Improving community understanding of medical research: audience response technology for community consultation for exception to informed consent.

INTRODUCTION: The Department of Health and Human Services and Food and Drug Administration described guidelines for exception from informed consent (EFIC) research. These guidelines require community consultation (CC) events, which allow members of the community to understand the study, provide feedback and give advice. A real-time gauge of audience understanding would allow the speaker to modify the discussion. The objective of the study is to describe the use of audience response survey (ARS) technology in EFIC CCs.
METHODS: As part of the Rapid Anticonvulsant Medication Prior to Arrival Trial (RAMPART), 13 CC events were conducted. We prepared a PowerPoint™ presentation with 4 embedded ARS questions,according to specific IRB guidelines to ensure that the pertinent information would reach our targeted audience. During 6 CCs, an ARS was used to gauge audience comprehension. Participants completed paper surveys regarding their opinion of the study following each CC.
RESULTS: The ARS was used with minimal explanation and only one ARS was lost. Greater than 80% of the participants correctly answered 3 of the 4 ARS questions with 61% correctly answering the question regarding EFIC. A total of 105 participants answered the paper survey; 80-90% of the responses to the paper survey were either strongly agree or agree. The average scores on the paper survey in the ARS sites compared to the non-ARS sites were significantly more positive.
CONCLUSION: The use of an audience response system during the community consultation aspects of EFIC is feasible and provides a real-time assessment of audience comprehension of the study and EFIC process. It may improve the community's opinion and support of the study.

INTRODUCTION

There is a critical need for research in certain emergency medical conditions to improve outcomes. Given the acute nature of some of these conditions, such as sudden cardiac arrest, strokes, and status epilepticus, obtaining an informed consent may be impossible.1,2 To address this ethical issue, the Department of Health and Human Services (DHHS) and the Food and Drug Administration (FDA) described guidelines in 1996 to allow research to be carried out with an exception from informed consent (EFIC).3 The federal policies require that the research subject be in a life-threatening condition for which available modalities of treatments are thought to be unsatisfactory. The potential subject must be unable to consent and there is no time to contact the legal representative. For the subject to take part in the study, there must be a possibility that the subject will benefit.4–5 The federal regulation that governs EFIC – 21 CFR 50.24 – requires that both community consultation (CC) and public disclosure (PD) occur in that local community and that local institutional review board (IRB) approval be obtained prior to EFIC activities and enrollment of subjects.6 Public disclosure requires that researchers inform the community that a study will be taking place, usually done through mass media, such as the internet, television and radio advertisements. Community consultation is designed to allow members of the community and stake holders to understand the study, provide feedback and act in an advisory role.7 It is also an opportunity to hear concerns, suggestions and questions that may have not been considered. An important part of community consultation is to sufficiently educate the community members about the study so that they are able to provide meaningful feedback.2 A lack of understanding may complicate the evaluation of community feedback by the investigators and IRBs. It can be difficult for the speaker to gauge the level of audience understanding. A real- time gauge of audience understanding would allow the speaker to modify the presentation, reiterate important aspects of the study, and ensure appropriate communication. Several studies have been published on the role of community consultation and how the general population views them, but we have not found evidence that demonstrates that the community groups understand the proposed study.

An audience response system (ARS) is an electronic device that creates an interaction between the presenter and his audience in real time and allows for immediate feedback. ARS have been playing an important role in medical education and have been shown to have more student appeal and satisfaction than didactic sessions in continuing medical education learners.8,9 This paper describes the novel use of audience response systems in community consultation for EFIC.

METHODS

Study Design

This study describes the use of an ARS during EFIC community consultations for a single site of the RAMPART study.10 We retrospectively compared ARS and non-ARS CC sites for potential differences in baseline demographic characteristics and average rank scores of the final paper survey. Our IRB approved this study as a part of the RAMPART EFIC process.

Study Setting and Population

During the EFIC process, our site carried out 13 CC events in the spring of 2009 and winter of 2010 in the local metropolitan area. CCs were organized with various community groups in the greater metropolitan area, such as churches, schools, and support groups. Six of the community consultations used the ARS presentations. The ARS and non- ARS consultations were chosen based on convenience.

Study Protocol

We prepared a PowerPoint™ presentation with audience response technology from Turning Point Technologies, Inc (Youngstown, OH) according to specific IRB guidelines to ensure that the pertinent information would reach the targeted audience. We embedded the slide set with 4 ARS questions that were thought to address the most important issues a community needs to understand prior to the study’s start (Figure 1). At the ARS sites, we used the pooled answers to the embedded questions to guide any necessary additional explanations of the study during the presentation. The same slide presentation was given to all of the ARS consultations.

Figure 1

Audience response questions.

At the end of all of the CCs, an anonymous final written survey was completed by individuals to obtain feedback from the audience and to gauge whether the audience was supportive of the study and willing to participate in it. The survey collected basic demographic information, qualitative comments, and had 4 questions related to opinions about the study with a Likert scale (Figure 2). Not all of the CCs were amenable to a slide presentation and the ARS slide set was not used there. Also, every participant did not complete a final paper survey.

Figure 2

Paper survey questions regarding community members’ opinion of study.

Data Analysis

We reported age as mean and standard deviation in both groups and compared it using a 2-sided 2-sample t-test. Gender and ethnicity were reported as frequency and percent for each category and compared between groups using a Chi-square test and a Fisher’s exact test, respectively. We evaluated educational level using a 2-sided Cochran-Armitage trend test to determine whether or not there was a significant trend in education level when compared between the 2 groups. The 4 questions related to opinions about the study were assigned ranked values from 1 to 4 (1: strongly disagree, 2: disagree, 3: agree, and 4: strongly agree). The average rank scores for ARS and non ARS were reported for each question and compared between groups using 2-sided Wilcoxon rank-sum tests.

RESULTS

The investigators presented information on the RAMPART study to 13 groups during the CC process. At 6 of these meetings, the presentation was augmented by the use of ARS to understand audience comprehension. The ARS process required minimal explanation, and only 1 unit was lost. There was a significantly higher average age and lower proportion of females in the ARS group compared to the non-ARS group. There was no significant difference in ethnicity or educational level between the 2 groups (Table 1).

Table 1

Demographics of community members attending a community consultation event.

	ARS sites	Non ARS sites	p-value
Participants	76	29
Mean age	49.9 ± 14.2	41.8 ± 13.8	0.011
Female	38 (49%)	22 (76%)	0.014
White	21 (28%)	6 (21%)	0.276
Hispanic or Latino	0 (0%)	1 (3%)
African American	46 (61%)	22 (76%)
Asian	5 (7%)	0 (0%)
Pacific Islander	1 (1%)	0 (0%)
Native American	0 (0%)	0 (0%)
Other	3 (4%)	0 (0%)
No school	0 (0%)	0 (0%)	0.296
Elementary	0 (0%)	(0%)
Some high school	2 (3%)	4 (14%)
High school	7 (12%)	3 (10%)
Some college	21 (35%)	8 (28%)
College	30 (50%)	14 (48%)

ARS, audience response system

During the ARS presentations, greater than 80% of the participants correctly answered 3 of the 4 questions. However, the question referencing EFIC and enrollment without the patient’s informed consent was correctly answered 61% of the time. The ARS results for these 6 groups are shown in Table 2.

Table 2

Audience response data summary.

Question	% Correct (Total N)
Which of the following are risks of the study?	81% (80)
We will start the study without the consent of the patient.	61% (80)
Which of the following people will NOT be enrolled?	92% (86)
Can I opt out of the study?	93% (87)

The final paper surveys were intended to obtain written feedback from the community regarding the study. Overall, the survey respondents had an 80–90% favorable response (strongly agree or agree) to the questions presented. Agreement scores were significantly higher in the ARS sites compared to the non-ARS sites for 3 of the 4 questions (Table 3).

Table 3

Paper survey question with results for sites that used audience response versus those that did not.

Question		Strongly agree (4)	Agree (3)	Disagree (2)	Strongly disagree (1)	Average score	p-value
This study offers more benefit than harm for the seizing patient	ARS sites	36 (47%)	36 (47%)	3(4%)	1(1%)	3.4	0.035
	76 (100%)
	Non ARS sites	7 (26%)	18 (67%)	2 (7%)	1 (4%)	3.1
	28 (96%)
I would be willing to participate in such a research study if I were having a serious seizure and unable to give my permission	ARS sites	30 (38%)	34 (47%)	9 (12%)	2 (3%)	3.2	0.661
	75 (98%)
	Non ARS sites	10 (34%)	14 (48%)	3 (10%)	2 (7%)	3.1
	29 (100%)
If a family member was enrolled in this study and I was told about it afterwards, I would agree to their continued participation, until they could consent for themselves	ARS sites	32 (42%)	42 (55%)	1 (1%)	1 (1%)	3.4	0.010
	76 (100%)
	Non ARS sites	5 (17%)	21 (72%)	1 (3%)	2 (7%)	3.0
	29 (100%)
Are you supportive of this study being done in your community?	ARS sites	43 (57%)	30 (39%)	2 (3%)	1 (1%)	3.5	0.024
	79 (100%)
	Non ARS sites	8 (30%)	20 (67%)	0 (0%)	1 (3%)	3.2
	29 (100%)

ARS, audience response system

DISCUSSION

Since the ARS technology was invented in the 1960s, it has been used in evaluating the response of large audiences. As the technology advanced, its application increased to cover a broad range of industries and organizations, such as marketing, corporate training, game shows, universities and continuing medical education.9 Miller et al evaluated the role of ARS for the continuing education of health professionals and found that ARS enhanced audience attention and learning.8 Homme et al concluded that ARS significantly increased attendance and participation during weekly residency board review courses, and the residents perceived that the experience was educational.11 During the RAMPART CCs, ARS was found to be a feasible method to measure the audience’s understanding of the presentation. From a financial aspect, it may be a cost-effective investment to ensure the message is understood. Furthermore, ARS is an easy way to automatically capture data, allowing the researcher to bypass the tedious data entry process that is necessary when written surveys are used. ARS also works for various size audiences. It is convenient for large audiences and also allows for anonymous feedback in small groups.12

The ultimate goal of community consultation is to provide the audience with an advisory role through which they can channel concerns, questions and feedback to the IRB and the study investigators.13 Based on their feedback, the IRB and investigators can modify some aspects of the protocol to protect the potential subjects. In a 2007 review article on CCs, Baren et al asked if the community grasped this advisory concept.14 While most of the literature addresses how CCs should be organized, and how much information should be divulged during the event, no literature has addressed the issue of audience comprehension. 15 The local community enrolled in this study was primarily composed of minorities and African Americans. This subset of the population has been shown to be reluctant to participate in clinical trials due to previous experience with research, such as the Tuskegee experiments.16 Through the use of ARS, the speakers were able to identify that 40% of the audience at the CCs did not understand the concept of EFIC and that the study would be started without informed consent. Therefore, the presenters were able to review and reiterate this information. This ultimately may have led to a greater understanding of the presentation’s content and overall purpose of the study and more support from the community. Alternatively, the ARS group may have felt more engaged and therefore more likely to support the study. For a different study, better understanding may possibly lead to less support if there are concerns with the study protocol.

To make the CCs more credible to the audience, the presentations were given by a physician. While a physician would have the knowledge to answer most of the audience’s questions, he/she can also be a source of intimidation to a lay person. While questions were encouraged during CCs, a lay person may be reluctant to ask simple questions to a physician. Similarly, given the large size of audience at CCs, an individual person might be reluctant to voice an opinion in such a large forum.14 However, through the use of the ARS, the audience has an anonymous method to voice their opinion.12, 13

LIMITATIONS

There are many limitations to this study. We did not prospectively design the CCs to compare ARS and non-ARS sites. Therefore, the 2 groups are not matched cohorts and we cannot draw definitive conclusions regarding their comparison. Also, we did not assess understanding of the study during the final paper survey; therefore we cannot comment on the effectiveness of either presentation with regards to comprehension.

Also, not all of the participants completed a paper survey, and everyone at the ARS sites may not have answered the ARS questions. Therefore, we do not have data to determine the overall response rate for the ARS presentations or the final survey.

Due to the anonymous nature of the paper surveys and the ARS responses we could not directly compare the responses of individuals.

CONCLUSION

The use of an audience response system during the CC aspects of EFIC is feasible, and provides a real-time assessment of audience comprehension of the study and EFIC process. It may improve the community’s opinion and support of the study. This article will hopefully inspire future research in improving community consultations.