Eliciting perspectives of the key study population: An effective strategy to inform advertisement, content and usability of an online survey for a national investigation.

This short communication demonstrates how conducting a focus group with members of a proposed study population of interest can help inform the suitability of study interventions and appeal of recruitment strategies prior to study commencement. People living with HIV (PLWH) were recruited to participate in a focus group to elicit feedback on the content and design of an online survey on sweetener knowledge and consumption; and usability of the survey which was ultimately launched nationally. Ten participants (age 55 years, 60% male, 70% non-Caucasian, 60% income < $25,000) attended and rotated through three stations and completed questionnaires to evaluate survey content, advertisement imagery and taglines, and ability to access and navigate the survey platform. Participants also engaged in open dialogue to discuss potential community and web-based recruitment strategies familiar to PLWH. Findings from the focus group helped investigators identify and select advertisement and recruitment strategies that were appealing to PLWH, refine and improve clarity/layout of the survey content, and enhance usability of an online survey intended for PLWH in the United States, age 18 years and older. Prospectively engaging individuals from a key study population in the early phase of study development is an effective strategy to assist in the development of study interventions and recruitment/advertisement materials designed for a specific population.

Introduction

Utilizing focus group methodology to receive feedback from representative members of a proposed study population of interest can prove valuable to assist with the development of effective recruitment materials as well as with usability of investigational tools. Despite the widespread use of focus groups in health research there are few detailed accounts utilizing this methodology to customize recruitment materials and inform development of investigational tools among people living with HIV (PLWH). This short communication shares findings from a focus group conducted among representative members of this key population prior to launching an investigator-initiated, national online survey aimed to assess sweetener knowledge and consumption in PLWH, and relationships of social determinates of health with these variables. The goal of the focus group was to solicit perspectives on recruitment materials and outreach strategies, as well as gain insights from this key population on survey content, clarity of questions, and length and usability of an online survey.

Methods

Ethics statement

This study was approved by the Partners Human Research Committee. Consent was implied by participation in the focus group.

Participant recruitment and eligibility

People living with HIV (by self-report) age 18 years and older were eligible to participate in the focus group. Volunteers were recruited via flyers distributed at HIV community organizations and health centers throughout Greater Boston, as well as advertisements on Craigslist. Recruitment was monitored by study investigators to ensure equal representation by participant sex, race, and education level.

Procedures

The focus group was held in October 2018. At the beginning of the session, investigators shared an overview of the purpose and procedures of the focus group. Participants completed a questionnaire that collected demographic and clinical data, and data related to education, income, food security and housing. Next, each participant rotated through three stations monitored by the study team to evaluate: [1] survey content, [2] usability of the online survey, and [3] advertisement imagery and tagline. At each station participants evaluated questionnaires developed by the investigators and comprised of multiple choice and open-ended questions. Participants rated ease of accessing the online survey, usability of the online survey and content, comprehension of instructions to complete the survey, and ease of navigation and completion. Participants were also asked to read survey questions and identify those that were [1] confusing or unclear, [2] inappropriate, [3] repetitive, or [4] missing. The survey includes demographic, social and clinical questions, as well as components of the NHANES Food Frequency Questionnaire [1], the Rapid Eating Assessment for Patients [2], the 2014 FDA Health and Diet Survey [3] in addition to questions on sweetener knowledge [4]. For the advertisement evaluation, three advertisement options were provided (Fig. 1A; advertisements were collaboratively designed by the investigators and a graphic designer). Participants were asked to rank preferred choices for each advertisement imagery option, font type, and tagline by selecting (A) really like, (B) like, or (C) don't like. Upon completion of the three evaluation stations, all participants reconvened to engage in an open dialogue with the investigators about recommendations for study advertisement venues such as websites, magazines, and local and national community organizations serving PLWH, including women and those of diverse racial and ethnic backgrounds. Participants received a meal during the focus group, remuneration, and transportation if needed.

Fig. 1

Evaluation and development of national online survey advertisements based on input gleaned from focus group participants. (A) illustrates imagery, tag lines, and font options that were evaluated. Focus group participants were asked to rate each font type, image and tag line, separately. (B) Final advertisements developed based on input from focus group.

Data analysis

All statistical analyses were performed using JMP Pro 13. Descriptive statistics were computed to analyze the questionnaire data. Continuous measured outcomes are presented as mean ± SD values. All categorical variables are reported as proportions. Open comment data obtained through comments written on the survey by participants and from field notes taken during the session were reviewed and compiled by the investigators. Collectively, all data were used to refine the online survey content, improve usability and determine advertisement type and outreach strategies.

Results

Participant characteristics

Twelve individuals were invited to participate in the focus group, and 10 could attend (1 Hispanic male, and 1 White female could not attend). Characteristics of the focus group participants are shown in Table 1. The mean age of participants was 55 ± 7, 40% were female and the majority were non-White (70%). The mean duration of HIV infection was 20 ± 9 years and almost all participants reported having an undetectable HIV viral load. Overall, participants had variable social and economic characteristics, with greater than half having at least a high school diploma and the majority reporting income <$25,000. Most participants reported stable housing and access to fresh fruits and vegetables some or all the time.

Table 1

Characteristics of focus group participants.

Demographic characteristics
	Total (n=10)	Male (n=6)	Female (n=4)
Age (years), M ± SD	55 ± 7	55 ± 6	56 ± 9
Race
White	3 (30)	3 (50)	0 (0)
Black	5 (50)	3 (50)	2 (50)
Asian	0 (0)	0 (0)	0 (0)
Native Hawaiian or other Pacific Islander	0 (0)	0 (0)	0 (0)
American Indian or Alaskan	0 (0)	0 (0)	0 (0)
More than one race	1 (10)	0 (0)	1 (25)
Other	1 (10)	0 (0)	1 (25)
Ethnicity
Hispanic or Latino/Latina origin	0 (0)	0 (0)	0 (0)
Education Level
Grade School	0 (0)	0 (0)	0 (0)
High School Diploma	6 (60)	2 (33)	4 (100)
Associate Degree	2 (20)	2 (33)	0 (0)
Undergraduate Degree	1 (10)	1 (17)	0 (0)
Graduate (Master) Degree	1 (10)	1 (17)	0 (0)
Doctorate Degree	0 (0)	0 (0)	0 (0)
Yearly Income (2 missing)
$0 dollars	0 (0)	0 (0)	0 (0)
$1 to 9999	4 (50)	1 (25)	3 (75)
$10,000 to 24,999	2 (25)	1 (25)	1 (25)
$25,000 to 49,999	1 (12.5)	1 (25)	0 (0)
$50,000 to 74,999	1 (12.5)	1 (25)	0 (0)
$75,000 or greater	0 (0)	0 (0)	0 (0)
HIV-related parameters
Duration HIV (years), M ± SD	20 ± 9	20 ± 12	20 ± 6
CD4+ T cell count at last medical appointment, M ± SD	893 ± 233	740 ± 210	1008 ± 195
Viral load undetectable (1 missing)	9 (100)	5 (100)	4 (100)
Measures of food securityIn the proof version, these all appear on 2-3 lines, the first line is indented but the 2nd and 3rd lines are not, making them not aligned with the first and making it appear messy and difficult to read. Please make sure lines 2 and/or 3 are aligned with the first
In the last 12 months, ate less because there wasn't enough money for food (YES) (1 missing)	3 (33)	3 (60)	0 (0)
Access to fresh fruits and vegetables all the time (YES) (1 missing)	6 (67)	4 (80)	2 (50)
Refrigerator at home to store food (YES) (1 missing)	9 (100)	5 (100)	4 (100)
Stove top to cook food (YES) (2 missing)	7 (88)	3 (75)	4 (100)
Oven to cook food (YES) (1 missing)	8 (89)	4 (80)	4 (100)
Microwave to cook food (YES) (1 missing)	7 (78)	3 (60)	4 (100)
Worried about having stable housing in the next 2 months (YES) (1 missing)	1 (11)	1 (20)	0 (0)

Note. All data are reported as n (%) unless otherwise indicated. SD = standard deviation.

Survey content and usability of the online platform

Based on participant feedback, many survey items were restructured, and the layout and presentation of the survey content and response items were enhanced to promote clarity and flow. Table 2 shares participant responses for usability of the online survey. Although participant responses varied, more than 50% found using the online survey was “easy” for all functionalities except for accessing survey from the online advertisement. A frequent suggestion recorded in the field notes was to change the selection item at the end of the survey from “save” to “submit” to indicate completion of the survey. Additionally, to include logic within the survey to indicate if a survey question was missed, and if so, which question was missed.

Table 2

Evaluation of online survey usability reported by focus group participants.

	Total (n=10)
Reading and understanding survey instructions (1 missing)
Easy	6 (67)
Pretty Easy	3 (33)
Hard	0 (0)
Accessing survey from advertisement
Easy	3 (30)
Pretty Easy	4 (40)
Hard	3 (30)
Reading survey content online
Easy	7 (70)
Pretty Easy	1 (10)
Hard	2 (20)
Clicking or selecting answer box (1 missing)
Easy	8 (89)
Pretty Easy	1 (11)
Hard	0 (0)
Scrolling down to get to other sections of the survey (1 missing)
Easy	7 (78)
Pretty Easy	2 (22)
Hard	0 (0)
Realizing that you have finished the survey (1 missing)
Easy	5 (56)
Pretty Easy	3 (33)
Hard	1 (11)

Note. All data are reported as n (%).

Advertisement

The tagline “How Sweet Are You?” and the pink swirl image were the most frequently endorsed options in the advertisement review questionnaire (Fig. 1A). However, during the open dialogue, participants reported that they also preferred a second advertisement image, the image that was used on the recruitment flyer for the focus group. Based on this feedback, two final advertisements were selected (Fig. 1B). Finally, focus group participants shared recommendations for advertisement venues, including online publications and newsletters, dating sites, and suggested that study flyers and postcards be available at HIV community organizations.

Discussion

Outcomes from this focus group among PLWH, the key study population for a subsequent online national survey study, demonstrate that conducting this was a useful strategy to: [1] help investigators identify and select advertisement and recruitment strategies that were appealing to the proposed study population; [2] reduce, refine, and improve clarity/layout of the survey content to be used in a subsequent study; and [3] enhance usability of an online survey. Including stations with topic-specific questionnaires in conjunction with open dialogue allowed for a targeted approach that yielded both objective and subjective perspectives from the participants, and ultimately, provided comprehensive guidance for the investigators regarding the study survey, advertisement, and ideas for future recruitment.

Recruitment, retention and fidelity to study interventions critically influence study outcomes. Thus, prioritizing these factors and eliciting insights from the experts-the specific population to be investigated- is an essential first step. Prior studies among PLWH have pro-actively sought perspectives on barriers and facilitators to research participation, study advertisement, and pre-trial preparation [5,6] and findings have enhanced the development of patient-centered recruitment and study implementation strategies. Findings from these studies also helped to inform evidence-based strategies for recruitment of special patient populations that are often under-represented in research studies, including women [5,6]. The focus group conducted in our study was not intended to identify strategies for recruiting specific populations of PLWH, but rather to identify strategies to most effectively design, recruit, and implement a consumer-friendly online survey at a national level. Given that the use of a national online survey for data collection is a method few prior HIV studies have utilized [7], partnering with PLWH through the focus group was instrumental to informing the survey content, advertisement imagery, and advertisement strategy. Importantly, since the survey is the primary data collection tool in the subsequent national study, testing usability of the online survey to identify and address concerns related to survey use and completion described by prospective participants helped to ensure that the final product was vetted.

Although the focus group yielded rich insights and suggestions to enhance the online survey and advertisement, there were limitations. The focus group was small, less than half of the participants were women, and all participants resided in Massachusetts. Further, most participants had at least a high school diploma and few had concerns regarding housing and food security, which may not be representative of the sample that will complete the subsequent national online survey. Therefore, when conducting a focus group to inform a similar intervention, it may be beneficial to elicit more detailed social information from prospective participants during the screening process to broaden the socioeconomic diversity of the focus group sample.

Conclusions

Prospectively engaging individuals from a key study population in the early phase of study development may be an effective strategy to help investigators determine appropriateness, clarity, and usability of study interventions designed for a specific population. Furthermore, gleaning input and perspectives from the intended study population prior to study initiation may enhance engagement and enrollment, expedite study completion, and may yield more meaningful results.

Author contributions

Study design (K.V.F, S.E.L), data collection (K.V.F, S.E.L., E.K.), data interpretation (K.V.F., S.E.L., E.K.), drafting of manuscript (K.V.F., E.K., S.E.L.), critical review of manuscript (K.V.F., S.E.L., E.K., C.R.).

Declaration of competing interest

Kathleen V. Fitch, MSN has received support from an educational grant from unrelated to this project. Sara E. Looby, is a non-paid board member of the non-profit organization Healing Our Community Collaborative, Inc. Boston, MA, and has received 1-time speaker fees from the Association of Nurses in AIDS Care in 2017 and 2019, unrelated to this project.