Utilizing Multilingual Methods and Rapid Analysis for Global Qualitative Research During a Pandemic.

Historically, qualitative research has complemented quantitative biologic and epidemiologic studies to provide a more complete understanding of pandemics. The COVID-19 pandemic has generated unique and novel challenges for qualitative researchers, who have embraced creative solutions including virtual focus groups and rapid analyses to continue their work. We present our experience conducting a multilingual global qualitative study of healthcare resilience among teams of pediatric oncology professionals during the COVID-19 pandemic. We provide an in-depth description of our methodology and an analysis of factors we believe contributed to our study's success including our use of technology, engagement of a large multilingual team, global partnerships, and framework-based rapid analysis. We hope these techniques may be useful to qualitative researchers conducting studies during the current pandemic, as well as for all pediatric oncology studies including multiple languages or geographically disparate subjects.

Introduction

The COVID-19 pandemic has changed how individuals interact with their communities, how patients interact with healthcare systems, and how investigators conduct research. While many of the initial studies on COVID-19 reported quantitative analyses, more recent qualitative work has explored the perspectives of healthcare providers (Liu et al., 2020; Munawar & Choudhry, 2020) and the impact of the pandemic on patients with chronic diseases (Gharzai et al., 2020; Giebel et al., 2020), including pediatric cancer (Graetz et al., 2021a, 2021b, 2021c; Saab et al., 2020; Vasquez et al., 2020). During previous pandemics, qualitative research has complemented quantitative biologic and epidemiologic studies to allow for a more complete understanding, particularly exploring social responses (Teti et al., 2020). However, there are unique challenges to conducting qualitative research under those circumstances.

Classically, qualitative methods rely on personal data collection including direct observation, interviews, and focus groups. Early in the course of the COVID-19 pandemic we learned the importance of social distancing to reduce the spread of the virus. To continue conducting qualitative research safely, researchers have adapted and emphasized virtual data collection (Dos Santos Marques et al., 2020). Thus far, these methods have been more frequently utilized in high-income countries that have consistent internet access and easy access to technology. Following data collection, rapid analysis has been embraced to share qualitative COVID-19 related study results in a meaningful and timely manner (Vindrola-Padros et al., 2020). In August 2020, we began a study to evaluate the impact of the COVID-19 pandemic on the care of children with cancer worldwide and successfully employed both techniques, including virtual focus groups in low- and middle-income countries, to accomplish our objective. By December 2020, we had completed 19 focus groups at 16 institutions in 16 countries using eight different languages. The speed with which we conducted this study enabled real-time reflection and dissemination of results during an ongoing pandemic. Here we present a detailed account of our study methodology and describe important lessons learned to guide qualitative researchers working during this pandemic and in the future.

Methods

Research design and participant selection

This work was conducted through the Department of Global Pediatric Medicine at St. Jude Children’s Research Hospital (SJCRH). It was the second part of sequential multiple methods study that sought to describe the impact of the COVID-19 pandemic on pediatric cancer care globally (Graetz et al., 2021a, 2021c). Participants for the qualitative cohort were purposefully selected based on data collected through a cross-sectional survey, the results of which have been previously published (Graetz et al., 2021a). The final survey question asked participants “would you like to tell us more about your institution’s experience during the COVID-19 pandemic?”. Only participants who answered “yes” and provided an email address were eligible for inclusion in the qualitative sample. Over 100 respondents met inclusion criteria. Of these, we purposefully sampled to include individuals from institutions that represented varying geographical contexts and resource settings. The World Bank assigns the world’s economies to four income groups (low, lower-middle, upper-middle, and high-income countries) and all were represented in our sampling (World Bank Country and Lending Groups—World Bank Data Help Desk, 2022). Furthermore, all regions as defined by the World Health Organization (“WHO | Definition of Regional Groupings,” 2017) (Americas, European, African, Eastern Mediterranean, South-East Asian, and Western Pacific regions) were included. Using answers from the survey, we selected institutions with a large volume of pediatric oncology patients (>50 new pediatric cancer patients/year) whose capacity to provide quality cancer care, including access to treatment modalities and changes in patient volume, had been impacted by the pandemic as defined by the quantitative assessment. Ultimately, we selected 16 different institutions (Figure 1).

Figure 1.

Map of included institutions. institutions at which focus groups were conducted are included, with countries identified by world bank income level.

Ethics

The institutional review board at SJCRH reviewed and approved the study with SJCRH identified as the coordinating center. Additional review and approval was conducted by regulatory bodies and ethics committees at individual sites as required. All participants provided verbal consent for participation and recording.

Local principal investigator and focus group selection

A principal investigator (PI) was identified at each participating institution. Local PIs were responsible for explaining the study to their institution, obtaining institutional review board approval if necessary, and recruiting focus group participants at their institution. Local PIs were selected based on survey responses, and all included institutions that had previously established relationships with SJCRH and were members of the St. Jude Global Alliance (Home St. Jude Global, 2022). Focus group participants included interdisciplinary professionals who provide care for children with cancer or were otherwise involved in the institutional response to the COVID-19 pandemic. While focus groups conducted with healthcare professionals are sometimes structured to be homogenous by discipline, we felt it was important for this study to include pediatric oncology professionals with diverse perspectives and roles who had been brought together to work in teams during the pandemic. Local PIs recruited participants to ensure adequate representation from these different viewpoints. Roles varied by site and included pediatric oncologists, infectious disease specialists, intensivists, surgeons, emergency room physicians, pathologists, palliative care specialists, hospital administrators, nurses, patient advocates/non-governmental organization (NGO) directors, social workers, pharmacists, psychologists, dieticians, and researchers (Table 1). Focus group sample size was not predetermined and ranged from 3 to 17 participants. All sites were given the option to host more than one focus group, allowing for separation of administrative staff from bedside staff as necessary to encourage honest conversation without fear of retribution. Local PIs decided if separate focus groups should be utilized at their institution, and at three institutions (United States, Philippines, Spain) participants were divided.

Table 1.

Focus group participants.

Institution Name (Total Participants)	Country	Focus Group Language	Physicians (n)	Nurses (n)	Other Roles (n)
Ain Shams University (8)	Egypt	Arabic	Pediatric oncologist (6)	Nurse (1)Nurse Supervisor (1)
Belarussian Research Center for Pediatric Oncology Hematology and Immunology (4)	Belarus	Russian	Infection control physician (1)	Head nurse (1)	Data manager (1)Head of infection control (1)
Cancer Diseases Hospital (10)	Zambia	English	Pediatric oncologist (1)	General nurse (2)Pediatric nurse (5)Palliative care nurse (1)		Surveillance officer (1)
Chris Hani Baragwanath Academic Hospital, University of the Witwatersrand (6)	South Africa	English	Pediatric oncologist (5)Oncology fellow (1)
Guangzhou Women and Children’s Medical Center (7)	China	Mandarin	Pediatric oncologist (2)Institution officer on COVID-19 (1)	Chief nurse (1)	Volunteer team lead (1)NGO officer (2)
Hopital Saint-Damien/NPFS (12)	Haiti	French	Pediatric oncologist (2)Emergency room physician (1)Pediatrician (1)Palliative care (1)Pediatric intensive care (1)Medical director (1)	Nurse educator (1)Head nurse (1)Infection control nurse coordinator (1)Infection prevention nurse coordinator (1)	Pharmacist (1)
Hospital Martagao Gesteira (8)	Brazil	Portuguese	Pediatric oncologist (2)Intensive care physician (1)Infectious diseases physician (1)Pediatrician (1)	Nurse director (1)Nurse (1)Infectious diseases nurse (1)
Hospital Pediatrico de Sinaloa (9)	Mexico	Spanish	Pediatric oncologist (4)Anesthesiologist (1)	Nurse (1)	Social worker (1)Epidemiologist (1)Psycho-oncologist (1)
Instituto Nacional de Enfermedades Neoplasicas (INEN) (12)	Peru	Spanish	Pediatric oncologist (7)Palliative care physician (1)BMT physician (1)Neuro-oncologist (1)	Chief nurse (1)Nurse (1)
Max Super Specialty Hospital (9)	India	English	Pediatric oncologist (3)Orthopedic surgeon (1)Intensive care physician (1)	Advanced practice nurse (1)	NGO trustee (2)Patient navigator (1)
Philippine General Hospital- Bedside Providers (7) Administrative Personnel (3)	Philippines	English	Pediatric oncologist (1)Pediatric oncology fellow (2)	Head nurse (1)	Database manager (1)Child life specialist (1)Patient navigator (1)
			Pediatric oncologist (1) Infectious diseases physician (1) Neurosurgeon (1)
Sardjito General Hospital (9)	Indonesia	Indonesian	Pediatric oncologist (2) Pediatrician (1) Pediatric intensive care physician (1) Pediatric infectious diseases physician (1)	Nurse (3)	Dietician (1)
St. Jude Children’s Research HospitalBedside Providers (6) Incident Command Center (8)	USA	English	Pediatric oncologist (2) Intensive care physician (1)	Advanced practice nurse (1)	Dietician (1) Psychologist (1)
			Incident commander (1) Lab director (1) Infection control physician (1)	Chief nurse (1)	Patient safety officer (1) Administrator (1) Quality manager (1) Patient experience (1)
The Indus Hospital (9)	Pakistan	English	Pediatric oncologist (3) Surgeon (1) Infectious diseases physician (1) Executive director (1)	Nurse (1) Infection control nurse lead (1)	Psychologist (1)
Uganda Cancer Institute (17)	Uganda	English	Pediatric oncologist (3) Radiologist (2) Executive director (1) Pathologist (1) Researcher (1)	Nurse (2)	Hospital administrator (2) Pharmacist (1) Administrator (1) NGO director (2) Researcher (1)
Vall d'Hebron Barcelona Hospital Group A (15) Group B (5)	Spain	Spanish	Palliative care pediatrician (1) Anesthesiologist (1) Pediatric surgeon (1) Hematologist (1)	Supervisor nurse (1) Nurse (2) Auxiliary nurse (1)	Foundation representative (1) Clinical trials coordinator (1) Volunteer coordinator (1) Educational psychologist (1) Psycho-oncologist (1) Cytogeneticist (1) Pharmacist (1)
			Pediatric oncologist (2) BMT physician (1) Infectious disease physician (1)		Social work coordinator (1)
TOTAL participants		164 total	88 physicians	37 nurses	39 other roles

Focus group logistics

Focus groups were conducted in the official language of the participating country to ensure participants would be able to fluently express their views and opinions without difficulty. This approach resulted in focus groups conducted in eight languages (Table 1). Due to considerations for time zone differences and work schedules during a pandemic, the team at SJCRH was flexible, and meetings were scheduled around participant availability, including weekend, evening, and early morning hours as we worked across time zones. Once participants were selected and invited, groups were created on a commonly utilized text-messaging application (Whatsapp) for each focus group to facilitate coordination and real-time troubleshooting day-of connectivity issues.

All focus groups were conducted virtually using an online video conferencing platform (Zoom or WebEx). For most focus groups, participants joined through individual links from their homes or offices to facilitate compliance with local social distancing recommendations. At some institutions, particularly in regions with inconsistent internet access, participants were physically together in a large conference room.

Participants were informed before the focus group that they would be asked to turn on their video cameras if possible, and we encouraged everyone to use the “gallery” or “grid” view so that they could see one another. This allowed for face-to-face communication, that in many ways was as intimate as an in-person focus group in which everyone is at the same table but not necessarily able to make eye-contact. We avoided using the share screen feature, even when reviewing focus group ground rules, to encourage participants to focus on one another rather than a presentation. The written chat function of the video conferencing platform was used for technology issues only to encourage fluid dialogue and enable us to capture all relevant focus group data on audio recording. One facilitator was tasked with troubleshooting technology concerns as they arose while the other focused on the study questions and group dynamics. In a few instances, participants without access to a microphone utilized the chat function to enter their responses, which were read out loud by a facilitator to capture the data on the recording.

Data Collection

A semi-structured focus group guide (Supplementary Material) was created based on seven factors previously described as important for resilience in healthcare (Iflaifel et al., 2020). These factors included: teamwork, in-situ practical experience, exposure to diverse views on patient’s situation, trade-offs, protocols and checklists, system design, and work arounds. The guide was written in English and iteratively revised by the investigator team. This guide was translated into Spanish and then back-translated and reviewed by bilingual members of the research team. Pilot focus groups were held in Spanish and English, and the interview guides were revised based on feedback from piloting. After this process, the English guide was translated into the additional languages by bilingual members of the research team. As available, a second bilingual member of the research team checked each translation.

Each focus group was moderated by two facilitators. Three core members of the research team conducted focus groups in English and Spanish. Additional facilitators who were native speakers of the languages needed for the non-English and non-Spanish focus groups were recruited from SJCRH. While the Department of Global Pediatric Medicine at SJCRH has many bilingual native speakers, we were concerned that pre-existing, sometimes hierarchical, relationships between department staff and study participants would affect comfort and honesty during the focus groups. We thus recruited bilingual staff from other departments at SJCRH. Our study team emailed a SJCRH listserv of postdoctoral fellows and approved interpreters asking for volunteer bilingual native speakers in six additional languages (Portuguese, French, Arabic, Mandarin, Russian, Bahasa Indonesia). Where possible, we recruited two facilitators for each language. Many recruited facilitators were bilingual and from the country in which the focus group was hosted, allowing for not only language but also cultural competence. Most volunteer facilitators had not previously conducted qualitative research. To address this, we organized formal facilitator training in which we briefly introduced qualitative research, reviewed the theory behind our study, and described the planned study methodology. This training was conducted virtually on the same video conferencing platform used for focus groups and included observed simulated practice facilitation Facilitators also attended pilot focus groups as observers. Following each focus group, volunteer facilitators were sent a link to an online survey to elicit feedback on their experience.

Focus groups were scheduled by local PIs in collaboration with the SJRCH investigator team. Each focus group was scheduled for 2 hours. Facilitators introduced themselves, set ground rules for the discussion that encouraged respect and diminished hierarchies, and had participants introduce themselves before beginning with the study questions. The shortest focus group lasted 45 minutes; most lasted 2 hours. Virtual focus groups were audio recorded. Audio-recordings were professionally transcribed and translated into English. Translated transcripts were reviewed and compared to the audio-recordings by bilingual members of the research team to ensure clarity and accuracy of translation. Focus groups were conducted from September through mid-October 2020 with transcription and translation beginning in mid-September. A timeline for the entire study process is depicted in Figure 2.

Figure 2.

Study timeline. timeline for study process from site selection through manuscript preparation.

Data analysis

This study was designed with a plan to utilize rapid turn-around analysis and a specific framework was used to structure the focus group guide and analysis templates. This framework was derived from the resilience in healthcare literature (Iflaifel et al., 2020) and structured around seven factors that have been demonstrated to facilitate resilience. At the time of study inception, the early qualitative work on COVID-19 focused largely on experiences and perspectives of healthcare professionals. We chose a resilience framework and utilized focus groups with the intention to explore positive systems-based solutions employed by teams of childhood cancer professionals across the world, in the hopes these expressions of resilience might be shared and adapted between settings. In addition, as topics raised had the potential to be sensitive to participants, we considered trauma-based interview techniques (Biddle et al., 2013) when designing our discussion guide.

Transcripts in English were de-identified and analyzed using rapid turn-around analysis (Qualitative Methods in Rapid Turn-Around Health Services Research, 2022). Analysis was done after the focus groups were completed and used the same healthcare resilience framework used to formulate the interview guide (Iflaifel et al., 2020). Prior to analysis, a table was made connecting factors in the model for resilience to interview questions with defined domains (Table 2). A transcript summary template was made using these domains (Supplementary Material). Four researchers participated in rapid qualitative analysis. All four team members iteratively tested the summary template using three transcripts. Based on this testing, domains were combined or added to minimize duplication and ensure the summaries would accurately capture as much content as possible. For analysis, transcripts were divided between the analysis team and summarized. Results from this analysis were compiled into matrices and presented as study findings.

Table 2.

Factors, domains and associated interview questions.

Factor	Question	Domain
Teamwork	How has COVID-19 affected or changed the way you work together as an interdisciplinary team?How has your team collaborated with other centers, either locally, regionally, or around the world?	Interdisciplinary teamsExternal collaboration
In-situ practical experience	Overall, what do you think about your hospital’s response to the COVID-19 pandemic?	Overall response
Exposure to diverse views on patient’s situation	How has COVID-19 affected or changed the way you work with families/patients?	Impact on families
Trade-offs	Of all the changes that have been made, are there any that you think will persist after this pandemic is over? Has anything changed for the better?	Positive strategiesPersistent changes
Protocols and checklists	Has your hospital instituted any new protocols or checklists to help implement new policies related to COVID-19?	Protocolized care
System design	How has your team been hearing about policy changes?Are there new hospital or unit/ward policies?	Communication of policy changesNew hospital policies
Work arounds	Suppose that you were in charge and could make one change to help your team during this time, what would you do?	Creative solutions

Lessons learned

The research strategy employed in this study was successful for quickly studying the perspectives of interdisciplinary pediatric cancer teams around the world during a pandemic. Our experience provides continued support for conducting qualitative research during this pandemic and demonstrates the potential effectiveness of virtual data collection and rapid analysis. Furthermore, we demonstrate success utilizing these strategies to quickly conduct global research in settings with limited resources and in many languages simultaneously. This work was enabled through our use of technology, the ability to engage a large multilingual team for conducting focus groups, global partnerships, and a framework-based rapid analysis focused on resilience in healthcare.

Use of technology

We found video conferencing to be an effective way to conduct international focus groups. Virtual focus groups have been prioritized by other qualitative researchers during this pandemic (Dos Santos Marques et al., 2020). In planning our approach, we had initial concerns regarding bandwidth and connectivity in resource-limited participating centers. Ultimately, technological and internet access issues did not significantly impact our data collection. Part of our success was due to preparation and flexibility. Specifically, local PIs coordinated with the clinical research associate at SJCRH to organize the focus group at a mutually convenient time.

The video conferencing platform we chose (WebEx) was one our research team was familiar with, and which was accessible to all participants via a shared access link. This platform was easily accessible from mobile devices and included a phone call-in link which enabled participation for those without computer access, and in cases of internet disruptions. For one focus group, we used an alternate video conferencing platform (Zoom) due to issues accessing the primary platform for participants in that country. Many of our participants had increased their use of web conferencing during the pandemic and this familiarity likely assisted their ability to participate in our study. When deemed necessary by local PIs, video conferencing test sessions were organized prior to the focus group. Careful planning to optimize the features of the video conferencing platforms and utilizing two facilitators with different tasks was key in the successful operation of focus groups.

Engaging a large multilingual team

Multilingual qualitative research can be challenging, time consuming, and resource intensive, but it is essential to improve our understanding of this global pandemic and its impact on pediatric cancer care. To appropriately evaluate a global phenomenon, we engaged with teams around the world and hosted focus groups in eight different languages. For this study, we prioritized language and cultural competence over previous qualitative research experience, allowing us to expand our research team and achieve our study objectives.

Following each focus group, volunteer facilitators were invited to provide their feedback through an online survey. Eight of the nine facilitators completed this survey. Of the respondents, 100% rated the experience as either “very” or “extremely” enjoyable and all said they would facilitate another focus group if given the opportunity. All facilitators were “extremely” or “very” satisfied with the facilitator training, facilitation guide, scheduling and communication with the study team. However, only 38% felt they had all the information they needed prior to the focus group and the rest stated they only had “some” or “most” of the necessary information. Suggested areas of improvement included the need for additional information about participants and sites prior to the focus groups; “I think it would be better to know more about who will participate in the focus groups and also about the hospital that these people work in before the meeting.” Facilitators also made suggestions regarding study design based on their experiences, “Another option could be one-on-one interviews, their technical logistics would be much easier to arrange.” One facilitator wondered about the cultural acceptability of the ground rule of “first-name basis,” suggesting “Other countries are very territorial of titles and ranks and if in the same focus group you have a nurse and a head of department it won’t fly good at all to ask them to be on first name basis.”

Unfortunately, most published research does not provide details regarding translation procedures (Regmi et al., 2010), and there are no standards for conducting or reporting qualitative research in more than one language (Alzyood et al., 2020; Squires, 2009). To maintain rigor, we employed piloting, used professional transcription and translations services, and conducted member checking. These elements allowed for reliability and validity even in the context focus groups being piloted in only English and Spanish. While we were able to collect data in multiple languages by training bilingual facilitators, we did not have the human resources to conduct analysis in multiple languages. We thus had audio-recordings translated and transcribed by a professional service we have worked with on previous multilingual qualitative research (Garza et al., 2021; Graetz et al., 2020, 2021b). To ensure validity, we had bilingual facilitators review transcripts and compare them to audio-recordings. When possible, we had a second bilingual facilitator review the first validation.

We acknowledge that not all investigators will have a large pool of available multilingual facilitators. However, our experience demonstrates that volunteers without prior qualitative research experience can be successfully trained for data collection and that multilingual data collection can be combined with unilingual analysis by a smaller team. We hope this encourages other researchers to aim for inclusivity by creatively harnessing the linguistic skills of partners and collaborators

Global partnerships

In addition to a large team of focus group facilitators, we partnered with local PIs at every included institution. These partnerships enabled us to structure our study around individual institutional needs and processes, and to ensure the study was conducted in a way that was relevant to the participants and culturally appropriate. Our history of successful collaboration fostered trust between researchers at SJCRH and international partners which was essential for rapid data collection during a time where everyone had many competing priorities.

We also maintained a degree of flexibility in our study design which enabled adaptation. We relied on local PIs to determine how many focus groups would be necessary at each institution and who would participate, with the composition of each focus group unique and specific to each institution. PIs helped us understand these needs, schedule focus groups, and determine if participants would join separately from home or together at the hospital via a shared link, depending on local regulations and preferences. Local PIs participated in the focus groups themselves. This eased logistics (e.g., PIs were able to ensure participants had protected time and arrived on time) and encouraged trust between other participants and focus group facilitators. Following data collection, local PIs reviewed the results of our analysis as a form of member checking (Candela, 2019). This also allowed PIs to reflect upon the situation at their own institution and learn from other participating hospitals. These individuals were included on the authorship team and reviewed all manuscripts published based on this work.

Framework-based rapid analysis

We utilized rapid analysis, which enables qualitative researchers to efficiently collect and analyze large amounts of qualitative data (Vindrola-Padros & Johnson, 2020). Given the nature of the phenomenon that we were seeking to describe, resilient healthcare for pediatric cancer care during a pandemic, the selected framework aided in a structured approach for data collection and analysis. Furthermore, the framework template allowed us to produce more concise data tables (Watkins, 2017), facilitating the reporting our findings in a timely matter. The conceptual framework domains in this evaluation were flexible enough to enable analysis within a single institution experience and across institutions with varying resource levels and geographical contexts, similar to prior reports (Skillman et al. 2019). Importantly, the analysis team piloted the template on a subset of transcripts and adjusted the domains slightly to ensure it could be used to appropriately capture and summarize the collected data. Findings from rapid analysis have been demonstrated to be consistent with in-depth analysis (Gale et al., 2019) and can expedite data analysis time (Taylor et al., 2018). Furthermore, this technique has been established as an effective way to conduct global qualitative research during the COVID-19 pandemic (Vindrola-Padros et al., 2020).

In our data analysis, audio transcripts were used due to the diverse set of languages employed for the focus groups and the need to consolidate in a single language (English) for the four team members utilizing the analysis templates. Given the different steps taken to ensure adequate translation for reliability and validity, we do not believe important points were lost by not using video recordings.

Our approach to these focus groups helped participants engage with our study and many local PIs reported that the experience of coming together through the focus group was therapeutic for their teams; “it was the first time the various parties involved in pediatric oncology at the Uganda Cancer Institute got together to exchange experiences...that is thanks to you” (PI from Uganda); “some of the reflections made us see all the work done with other eyes” (PI from Brazil). Matrices, which have been demonstrated to be helpful for structuring interventions with limited resources (Averill, 2002), were used to synthesize and communicate research findings.

Conclusion

Qualitative researchers, like other scientists, have found creative ways to adapt and adjust their work to the realities of the COVID-19 pandemic. We partnered with local PIs who had established relationships with SJCRH which engendered trust and facilitated study adaptation. This process taught us about our own institution’s potential for multilingual qualitative research and the power of engaging a global network to conduct culturally relevant, impactful qualitative research. In addition, we learned that virtual focus groups can be as effective as those conducted in-person. This process was cost- and time-efficient since the need to travel was eliminated. We are hopeful that our experience will inspire other qualitative researchers to consider the possibility of global, multilingual, virtual data collection both during this pandemic and beyond.