Integrating stakeholder feedback in translational genomics research: an ethnographic analysis of a study protocol's evolution.

PURPOSE: This study describes challenges faced while incorporating sometimes conflicting stakeholder feedback into study design and development of patient-facing materials for a translational genomics study aiming to reduce health disparities among diverse populations.
METHODS: We conducted an ethnographic analysis of study documents including summaries of patient advisory committee meetings and interviews, reflective field notes written by study team members, and correspondence with our institutional review board (IRB). Through this analysis, we identified cross-cutting challenges for incorporating stakeholder feedback into development of our recruitment, risk assessment, and informed consent processes and materials.
RESULTS: Our analysis revealed three key challenges: (1) balancing precision and simplicity in the design of study materials, (2) providing clinical care within the research context, and (3) emphasizing potential study benefits versus risks and limitations.
CONCLUSIONS: While involving patient stakeholders in study design and materials development can increase inclusivity and responsiveness to patient needs, patient feedback may conflict with that of content area experts on the research team and IRBs who are tasked with overseeing the research. Our analysis highlights the need for further empirical research about ethical challenges when incorporating patient feedback into study design, and for dialogue with genomic researchers and IRB representatives about these issues.

Introduction

In response to the persistent lack of diversity among genomics research participants,[1] a growing number of research funders are requiring researchers to recruit participants from backgrounds that reflect diversity across different dimensions, such as race and ethnicity, socioeconomic status, and primary language.[2] To do so effectively, researchers must ensure their studies are designed to be responsive to their participants’ needs.[3] For example, tailoring patient-facing materials like recruitment brochures and consent forms to the language and health literacy of the study’s target population may improve accessibility and support informed decision-making.[4-5]

Involving patient stakeholders in study design and development can make research findings more relevant to the health decisions these patients face, and thus more useful and likely to be taken up in practice—especially for research intended to address health disparities in a real-world setting.[6-7] This approach requires partnering with stakeholders with varied perspectives.[8] Patient feedback can help researchers determine appropriate outcome measures, design effective recruitment strategies, and address the ethical implications of different approaches to genomic results disclosure.[9-10] However, it may be necessary to make trade-offs when it comes to implementing feedback due to challenges such as feasibility and resources.[7]

This manuscript describes how we incorporated stakeholder feedback into research processes and participant-facing materials for the Cancer Health Assessments Reaching Many (CHARM) study. CHARM was designed to implement and evaluate the use of a streamlined approach to offering genomic sequencing for hereditary cancer risk. With decreasing costs and increasing options for genetic testing, particularly in clinical settings outside of academic medical centers, traditional approaches to genetic counseling, testing, and results disclosure are unlikely to scale in an equitable manner. CHARM’s streamlined approach—which includes web-based risk assessment and consent, testing on saliva samples, and phone-based results disclosure—is intended to improve access to and uptake of genetic testing among historically underrepresented groups. To ensure inclusivity of this approach, a robust patient stakeholder engagement process was built into the development of study processes and materials. Our focus in this manuscript is on the content and incorporation into the study of patient and other stakeholder feedback; implementation and evaluation of the patient stakeholder engagement process are described elsewhere. Through ethnographic analysis of study process data from the perspective of patient advisors, content area experts on the study team, and our institutional review board (IRB), we highlight several challenges for incorporating sometimes contradictory feedback into translational genomics research with diverse populations.

Methods

The CHARM study

CHARM is part of the Clinical Sequencing Evidence-generating Research consortium, a National Human Genome Research Institute, National Cancer Institute, and National Institute on Minority Health and Health Disparities research program of translational genomics studies aimed at developing best practices for implementing genomic sequencing among diverse and historically underrepresented patient populations.[2] In CHARM, primary care patients in two large healthcare systems are invited to take a self-directed, web-based family history risk assessment questionnaire, which comprises modified versions of two pre-existing cancer risk assessment tools,[11-12] to evaluate their risk of Lynch syndrome and hereditary breast and ovarian cancer syndrome. Those who are at high risk of one or both syndromes based on family history or who have insufficient family history information to make a determination of risk are invited to join CHARM, through which they undergo sequencing for cancer risk as well as optional additional testing for medically actionable secondary findings and/or carrier findings. Prospective participants review information about genetic testing and CHARM study procedures via a web-based tool but do not meet with a genetic counselor prior to testing. Participants receive their results by phone from a study genetic counselor using either a traditional or modified communication approach, and health records are reviewed to evaluate post-result healthcare utilization. Participants are asked to complete a baseline survey and two follow-up surveys, and a subset are invited to complete one or more qualitative phone interviews.

During the study start-up period, the interdisciplinary CHARM research team included over 50 investigators and staff at 9 different institutions. Due to the large size of the research team and project scope, many project tasks and responsibilities are divided across smaller topic-specific workgroups.

Development of study materials

A primary goal of the CHARM study is to include participants with limited health literacy and/or English proficiency, including individuals whose primary or preferred language is Spanish. To ensure accessibility, we drafted English study materials at approximately a 5th grade reading level, using simple sentence structure and plain language, and in a manner that would facilitate subsequent translation into Spanish. Project workgroups collaboratively drafted and reviewed materials and integrated feedback from patient advisors, as described below. The Spanish translation process was led by a CHARM co-investigator, a certified translator specialized in adaptation of health-related materials for individuals of limited literacy (NML).

Feedback from patient advisors

We engaged community feedback about study processes and materials in two ways. First, CHARM team members visited two classes at a local community college (one for adults learning to read and another for adults seeking their GED) to describe the CHARM study and obtain feedback on study messaging, recruitment approaches, and inclination to participate in genetic research. Second, we assembled two patient advisory committees (PACs), one each of individuals whose primary language was English or Spanish. We sought to recruit into the PACs individuals from groups historically underrepresented in genomic medicine, including individuals from ethnic and racial minority populations, non-native English speakers, and individuals from low socioeconomic backgrounds or with limited formal schooling. Clinicians and clinical researchers at study recruitment sites identified and referred potentially interested individuals, and study staff conducted brief screening interviews to confirm interest and availability.

The seven-member English-language PAC met four times in person as a group. Members were also invited to participate in four rounds of individual phone interviews. Because the ten Spanish-language PAC members’ work schedules made scheduling group meetings impossible, we obtained their feedback in four rounds of individual phone interviews conducted by native Spanish speakers on the study team. In each group meeting or interview, PAC members learned about the CHARM study and provided feedback on specific study processes (e.g., approaches to recruitment) and participant-facing materials (e.g., recruitment postcards, surveys), in roughly the order they would be encountered by study participants.

Interactions with patient advisors were designed to provide feedback on the study and materials, rather than as formal data collection activities. Therefore, group meetings were audio recorded to allow study team members who could not be present to listen to the conversation, whereas our goal of providing rapid feedback to the study team meant it was more efficient for phone interviewers to type detailed notes about each PAC member’s responses to each item on our structured list of questions, capturing direct quotes when possible. To share patient feedback with the CHARM workgroups whose materials or processes were discussed, a study team member reviewed recordings and/or notes from each group meeting or set of interviews, highlighting any consensus recommendations as well as agreements and disagreements in feedback. The CHARM stakeholder engagement workgroup reviewed these summaries and provided recommendations to the other workgroups.

Observational field notes

Study team members were trained by a PhD-level ethnographer (CM) to write field notes reflecting on study processes and challenges as they arose.[13] Reminders to submit field notes were sent approximately every two weeks during the study start-up period. From the start of the study in August 2017 through December 2018, 75 field notes were submitted by 23 different research team members, accounting for approximately half of the research team during that time period and representing multiple different workgroups, study sites, and individual roles.

Analysis

After collecting and incorporating feedback from patient advisors, obtaining IRB approval, and implementing the study, we sought to summarize how the perspectives of patient advisors, study team members, and IRB representatives had informed the study protocol, including what tensions and constraints arose when balancing these perspectives. We collected the patient advisory group meeting and interview summaries, 60 potentially relevant field notes discussing the development of enrollment processes and materials and/or the IRB review process, and correspondence between our study team and the IRB. Documents were sorted based on what part of the enrollment process they addressed and independently reviewed by two authors (SAK, DB). One author (SAK) drafted summaries based on the key points identified in these reviews. To minimize biased interpretations of the data, summaries were reviewed by two authors (DB, KS) who were not involved in the study start-up period, then discussed with the entire author team for additional input. Through these discussions, we identified the key issues that arose in each set of materials and common challenges that carried through multiple parts of the study.

Results

Recruitment materials and approaches: key issues

Patient advisors emphasized the importance of presenting clearly and upfront the potential benefits of getting genetic testing through the study to increase participation by historically underrepresented populations. They said recruitment materials should provide details about specific tasks and requirements for participation, including that participation would not cost them anything nor would they be charged later for test results. They also said materials should state that doctors approved of the research, because many people rely on their doctors’ advice regarding clinically relevant decisions. The IRB expressed concerns with emphasizing the potential benefits of genetic testing, framing free genetic testing as a benefit, and stating that clinical recommendations support genetic testing. They said describing the risk assessment as “important” and genetic testing as “recommended” could bias people toward participating. The study team disagreed, noting that these risk assessments are recommended care for individuals with a family history[14-16] and that not using this language could lead high-risk patients to inappropriately not seek out standard care. Ultimately, through discussions between the study team and IRB, the language in the recruitment materials was revised to focus on the potential benefit of talking to one’s doctor about genetic testing, rather than of genetic testing itself. Additionally, mentions of “free” testing were removed from recruitment materials except for a brief note at the end of the recruitment brochure that testing would be “at no cost.”

Regarding the recruitment approach, patient advisors viewed in-person recruitment as the most personal and likely to be effective, which was an approach used successfully on previous studies[17-19] and that the study team expected would increase the perception of the study’s legitimacy. In addition to reaching out to patients via email or mail, the study team planned to approach all patients within the eligible age range (18–49 years) who were present at each primary care clinic in which recruitment was to take place, with bilingual staff able to approach people in both English and Spanish. Patients would be introduced to the study and offered the opportunity to complete the family history risk assessment (and, if eligible, enroll in CHARM) while in clinic on a tablet or given a brochure so they could complete it later. However, given the study’s focus on diversity, the IRB advised that in-person recruitment by study staff might create the appearance of bias or profiling and undermine overall patient trust. They required that the study team not approach patients and instead indicated that the team could set up a booth where interested individuals could approach study staff, approving limited signage that would not reveal health information about the patient (i.e., no statements about what the study was about, including mentioning a family history of cancer). The study launched with this approach. While some patients approached the booth, the study team ultimately decided that passive in-person recruitment was ineffective and instead relied on other IRB-approved approaches (e.g., email, postcard).

Risk assessment tool: key issues

Application and user design engineers on the study team created a user interface for modified versions of the PREMM5[11] and B-RST 3.0[12] cancer risk assessment tools to screen for eligible CHARM participants. When reviewing the tool, patient advisors emphasized the value of clarity and transparency. They highlighted the importance of using straightforward language throughout and downplaying the names of the two tools as they would likely be confusing or meaningless to participants. There was ample discussion among patient advisors on what terms to use for some cancer types to maximize comprehension by readers of limited literacy (e.g., colorectal cancer vs. cancer of the gut, ovarian cancer vs. cancer of the eggs); some were unfamiliar with certain terminology and asked for explanations or definitions of unfamiliar terms (e.g., Lynch syndrome).

The study team had anticipated that many terms and concepts would be challenging to convey and made substantial efforts to revise the language in the questionnaire and results output both before and after obtaining patient feedback. Throughout this process, content area experts on the study team struggled with explaining complicated concepts to lay audiences in a way that felt precise and accurate (Table 1). The concept of risk of hereditary cancer syndrome based on the risk assessment results, for example, was challenging to convey in simple terms, as the team worried it could easily be confused with the risk of getting cancer and that lay interpretations of “high” or “low” risk might differ significantly from the intended meaning. Similarly, the IRB questioned how degree of risk would be reported to participants, requesting additional clarity from the team to ensure messages about risk were delivered in a manner that was appropriate for our target study population. This concept also raised questions among patient advisors, and there was not clear consensus in the English-language PAC on what term would best describe the concept of “risk” or “chance” of hereditary cancer syndrome; based on this feedback, the team decided on the term “chance of getting cancer”. The Spanish-language PAC offered different commonly used terms related to the concept of probability, ultimately landing on “chances o probabilidades” (“likelihood or probability”) as words that Spanish speakers from various socioeconomic and national backgrounds would understand.

Table 1:

Challenging concepts to convey to a lay audience

Concept	Early draft language	IRB-approved English language	IRB-approved Spanish language
Genetic testing	Genetic testing for cancer looks at your DNA to see if it has parts that we know make people more likely to get cancer	Genetic testing identifies changes in DNA that cause diseases that are passed down in families	La prueba genética identifica cambios en el ADN que causan enfermedades que se dan en familias
Mutation or variant	Gene change	Change in your DNA	Cambio en tu ADN
Positive result for hereditary cancer syndrome	If you get an abnormal result, that means your cancer risk is higher than with family history alone	If you get an abnormal result, that means you have a change in your DNA that increases your chance of getting one or more certain types of cancer	Si tu resultado no es normal, esto significa que tienes un cambio en tu ADN que sube tus chances o probabilidades de que te den ciertos tipos de cáncer
Elevated risk of hereditary cancer	More likely than other people to get cancer	Higher ch19:1ance of getting cancer than most people	Tus chances o probabilidades de tener cáncer son más altos que la mayoría de la gente
Medically actionable secondary findings	Genetic conditions that are related to your health that can be treated, prevented, or detected early	Health problems in you that may need medical attention	Resultados sobre tus problemas de salud que puedan requerir atención médica

Informed consent: key issues

The informed consent materials sequentially described the non-experimental, clinically available genetic testing being offered in the study, followed by the CHARM research activities. As with the risk assessment tool, patient advisors emphasized clarity and consistency throughout the consent materials, noting that inconsistency could provoke distrust of the researchers. They again highlighted the importance of presenting potential benefits in a direct and upfront manner, avoiding terms like “may” that could imply genetic testing is inaccurate and noting that most people who undergo testing will get reassurance from a negative result. The study team tried to incorporate patient advisor feedback in a way that balanced concerns about clarity and comprehension while maintaining an appropriate level of precision about the study. During its review, the IRB requested that the consent process include additional detail about the limitations of testing, potential risks of anxiety or uncertainty, and the possible test results.

Another key piece of feedback from patient advisors involved clarifying, as early in the process as possible, that prospective participants were being invited to join a research study. The risk assessment tool was designed as a clinical care quality improvement tool to identify potentially at-risk individuals. Study staff anticipated that most people who completed the risk assessment would not be eligible for the study. Thus, in addition to a clinical care activity, it served as a screening tool to determine eligibility. The study team therefore requested a determination of research status for the risk assessment as a quality improvement project that would not require all elements of a research informed consent, in contrast to the subsequent research activities. However, patient advisors commented that prospective participants completing a cancer risk assessment screener as regular care might feel it was a “bait and switch” when learning that a subsequent step (genetic testing) may be offered as part of a research study, and lose trust in the research team. The IRB shared similar concerns. They required the team to provide all elements of consent prior to determining eligibility for the study and for prospective participants to be informed, before completing the risk assessment, that there was a possibility of being offered testing (through either regular care or a research study) and being asked to join a research study.

Discussion

This analysis of how we integrated the perspectives of our patient advisors, study team members, and IRB into the CHARM protocol reveals several cross-cutting challenges that arose as we worked to ensure our study could achieve its goal of improving access to genetic testing: (1) balancing precision and simplicity in the design of study materials, (2) providing clinical care within the research context, and (3) emphasizing potential study benefits versus risks and limitations.

Balancing precision and simplicity

Consistent feedback from our patient advisors was that study materials should be simple, straightforward, and easy to understand, but content area experts on the study team struggled to describe complex genomics concepts without creating study materials that were overly challenging or overwhelming. These discussions and IRB feedback raised questions about the degree of detail necessary for informed consent, and how additional detail might influence participant understanding.

The tension between patient preferences and regulatory requirements regarding consent has been described elsewhere.[20-22] On one hand, straightforward and culturally appropriate language may support greater inclusivity of historically underrepresented populations in research,[4-5] which is a primary goal of CHARM and is critical for increasing the generalizability of genomics research.[1,3] In fact, some bioethics scholars question whether thorough comprehension of all regulation-required information is the best measure of the success of the consent process,[23-24] and neither the reading level of consent forms nor participants’ comprehension thereof are typically measured by IRBs.[25] On the other hand, it is important to ensure study materials are accurate and complete, and match potential participants’ literacy needs, to enable informed and values-consistent decision-making. Our patient advisors sought to represent the voices of the greater patient population, but their feedback may not have included all information that a prospective CHARM participant might find helpful or that content area experts might view as important to convey. This is also true in terms of our study’s implications for other research that seeks to include participants from various backgrounds, especially groups that are not well represented in CHARM. Similarly, IRBs at different institutions may have different interpretations of what information is ethically necessary than our IRB. As more research programs seek to include historically underrepresented groups, it will be important to consider carefully what language and information is most appropriate for communicating effectively with the participants they seek to include.

Providing clinical care within the research context

A second challenge arose from the relationship between the study’s research elements and the guideline-recommended[14-16] genetic testing that was offered as a part thereof, including whether it was appropriate to describe the test as recommended in the study enrollment materials and how this might affect inclusion of underrepresented groups. There is ongoing debate about the ethical oversight of clinically embedded research that blurs the research-clinical care boundary.[26] Prior work has suggested that IRBs struggle to regulate such research,[27] and that patients and IRBs may have different views about the best approaches to oversight.[20] In combination with this prior work, our analysis suggests there is a need for ongoing dialogue about this issue—particularly as it relates to the implications for health disparities research—and for greater consideration of the views of underrepresented patient groups.

Additionally, the feedback we received from patient advisors and the IRB alike emphasized the importance of clarifying the research nature of the study from the beginning of the enrollment process, to both preserve trust and ensure prospective participants adequately understood what they were being asked to do. In this case, our patient stakeholders and IRB were not in tension with each other, but rather challenged the research team’s initial perspective and highlighted the multiple purposes that a transparent message about the clinical-research distinction could serve.

Maintaining clinical standards for genetic testing and counseling also posed a challenge in relation to our use of a web-based consent process that did not include an in-person pre-test counseling visit. Although meeting with a genetic counselor has been part of the traditional approach to genetic testing,[14] the rapid growth of genomic medicine without corresponding growth in genetic counseling capacity has illuminated the importance of reassessing the feasibility and value of requiring face-to-face counseling in all cases, especially insofar as it can impede access to otherwise-desired testing.[28] Meeting with a genetic counselor may increase knowledge, reduce distress, and improve informed decision-making[29] and allow the counselor to conduct a risk assessment.[30] However, recent work has suggested that patients who do not have face-to-face pre-test counseling do not experience greater psychosocial distress[31] and that it may not be necessary for informed decision-making, although it may reduce decisional conflict among individuals with lower educational attainment or less experience with genetic testing.[32] As Ormond and colleagues argue, this evolving landscape suggests it may be time to consider alternative models to facilitate access among a broader population.[33]

Our team was interested in evaluating the effects of offering testing broadly, including to patients from historically underrepresented populations, on uptake of genetic testing, adherence to subsequent clinical recommendations, and health and psychosocial outcomes. Our streamlined approach to study enrollment incorporated key elements of genetic counseling into our self-directed risk assessment and informed consent tools; while these tools included resources for prospective participants to contact their doctor or a genetic counselor, they did not require a face-to-face discussion as part of the enrollment process. The IRB questioned this approach out of concern for potentially vulnerable individuals who might benefit from face-to-face counseling, raising a tension between inclusivity and protection of the individuals we sought to include. While our patient advisors did not raise concerns about our approach, it was not discussed explicitly, so we cannot conclude what they would have recommended. Prior work with patient stakeholders suggests that patients prioritize a “focused” and easily understood consent process in certain low-risk clinical settings,[33] but further dialogue with diverse patient stakeholders about these complex trade-offs, in addition to data like that being collected in CHARM, will be needed to inform the ongoing debate about this issue.

Emphasizing benefits versus risks and limitations

A third tension arose about what potential benefits and risks of genetic testing and research participation the study materials should describe and how those should be framed. Patient advisors felt it was important to be upfront about the study’s benefits, particularly its potential to make clinically recommended genetic testing more accessible to historically underrepresented groups, which aligns with other studies suggesting participants find value in understanding a study’s benefit to oneself[34] or others.[35] However, this perspective came into conflict with the IRB’s view that presenting the genetic testing offered as generally beneficial could cloud prospective participants’ judgment about enrolling.

This tension highlights the question of how research should be framed in recruitment and consent materials.[36-37] Dickert and colleagues argue that de-emphasizing a study’s benefits, while intended to protect participants, may actually reduce understanding and foster mistrust.[38] In the setting of efforts to reduce health disparities, positive framing can encourage uptake of recommended clinical interventions[39] and recruitment of diverse research populations.[36] Emphasizing study benefits does not seem to threaten the voluntariness of an individual’s decision through either undue inducement (i.e., participating because of an improper offer) or coercion (i.e., participating due to a threat of material or physical loss).[40] Additionally, studies suggest that framing research positively does not induce participation where an individual would not have otherwise wanted to enroll.[41-42] Future empirical studies should continue to examine any potential impact of positive framing on the voluntary nature of individuals’ decisions.

Limitations

This analysis highlights several challenges we faced in our study. A more comprehensive analysis of challenges faced across different types of studies would provide further detail about these tensions and how they might be resolved. While this single example highlights the perspectives of patient advisors, study team members, and IRB representatives involved in one study at a limited number of institutions, efforts to reduce health disparities in genomic medicine will require attention to these tensions across a broad range of individuals and institutions; our analysis provides a starting point for conversation but highlights a gap in understanding of these tensions in other settings.

Additionally, we approached this analysis from our position as members of the research team; while we recognize that this gives us a particular viewpoint on the study and have attempted to be transparent about our positionality, it is not possible to remove all sources of potential bias. To minimize bias, this manuscript was reviewed by research team members who were not part of the study design process, two members of our PACs (CR, PJ) who participated as co-authors, and two IRB personnel involved in reviewing the study.

Conclusion

Genomics research that aims to reduce health disparities can benefit greatly by incorporating the views of patient stakeholders, but integrating stakeholder feedback may be difficult if it comes into tension with the perspectives and priorities of content area experts on the study team and IRB interpretation of regulatory requirements. Our experience in the CHARM study highlights the need for further empirical research on the ethical implications of incorporating patient feedback into study design and materials, including gathering input from a broader range of stakeholders on appropriate language for study materials, evaluating the impact of a web-based consent process without face-to-face pre-test counseling on patient decision-making and psychosocial outcomes, and assessing how framing information affects voluntariness and decisional satisfaction. Additionally, researchers and IRB representatives must continue to discuss issues such as the tension between inclusivity and protection, in the context of specific studies and more broadly, to identify where their perspectives differ and work toward mutually satisfactory solutions. Finally, our work highlights the value of engaging with historically underrepresented patient stakeholders to ensure research is designed appropriately to meet the needs of the patients it seeks to include, as well as the limitations of incorporating patient perspectives into a study protocol. Involving patient stakeholders in a meaningful way requires capacity building that can be challenging within the time- and resource-pressures of a single study, especially when addressing complex issues that involve making trade-offs. If researchers are to incorporate patient values about complex questions into study design in a meaningful way, they will need to make long-term investments in diverse stakeholder engagement. While this process is neither simple nor straightforward, our experience suggests it has great potential to address the challenges confronting genomic medicine and move forward efforts to reduce health disparities.