Running an active gaming-based randomized controlled trial during the COVID-19 pandemic: Challenges, solutions and lessons learned.

Objectives: The COVID-19 pandemic has created significant obstacles for clinical trials and human subject research. This paper discusses the challenges our study team encountered while implementing an eHealth intervention during the pandemic, including: increased dropout, cancellation and rescheduling rates, increased mailing returns and delays, social distancing impediments, COVID-19 positive team members, and restricted training access. Study design: This is a short paper on research protocol for a six-month randomized controlled single-blind trial.
Methods: N/A.
Results: In response to these challenges, we changed the study protocol. We included multimodal communication models, amplified recruitment efforts, expanded our population's age range, increasingly utilized tracking labels, utilized external space for extra participants, and transitioned to a virtual RA training format. Conclusions: Sharing our experience and the adaptations required to run a clinical trial during the pandemic should provide useful and practical knowledge for institutions, funding agencies, and researchers. We believe that the lessons learned here would be applicable to future clinical trial research after the pandemic as well.

Introduction

The COVID-19 pandemic, and restrictions driven by public health considerations, created a structural barrier to maintaining a physically active lifestyle [1]. Specifically for children, the initiation of virtual schooling was followed by increased sedentary behavior and less sport and physical education engagement [2,3]. These negative health trends have been associated with accelerated weight gain in children, especially those age 6–11 [4]. Thus, already a longtime national public health concern, childhood obesity was exacerbated further during the pandemic, making more accessible exercise interventions necessary.

To address low childhood physical activity (PA), our team had planned an active video game intervention study since 2019. The protocol was fully detailed elsewhere [5]. The pandemic increased the importance of our ongoing work due to the further decrease in PA among children, as well as increased the relevancy of COVID-19 quarantine friendly eHealth interventions [4,6]. However, these novel implications came with the challenges of conducting human subject research during a pandemic. This paper discusses these challenges and what we modified to cope with them. The purpose of this paper is to inform those who are planning clinical trials of what we have learned to be effective protocol adaptations while using a pandemic-friendly intervention.

The COVID-19 challenges and adaptations to human subject research

Increased dropout, cancellation, and rescheduling rates

Our main trial was initiated on January 11, 2020 and all human subject research was suspended from March 17, 2020 until resumption on September 12, 2020. Approximately 61.5% of pre-pandemic participants were lost to follow-up after resumption. In addition, the dropout, cancellation, and rescheduling rates were higher. On average, across three visits, only 43% of participants presented without any rescheduling, and only an additional 13% of the total participants showed up after rescheduling. One primary reason may be attributed to the economic instability caused by the pandemic and the disproportionate economic impact on low socioeconomic status and minority populations [7], which made up most of our sample. Our sample had 19.7% of participants report a household income of less than $20,000 and an additional 28.2% report a household income of $20,000-$39,999. Due to financial constraints, families may not have been as available or capable of traveling to our study.

To improve attendance, we developed an enhanced multi-modal communication confirmation system and asked participants to provide their preferred method of communication. After research resumption, we sent all families scheduled for data collection a confirmation email 2–3 days before their visit, including a COVID-19 screening questionnaire and an attendance confirmation request (Fig. 1 ). If a participant neglected to respond to this email, text messages (or phone calls) requesting confirmation were sent 1–2 days before data collection to ensure communication with each participant. Utilizing text messages and phone calls proved useful, as 55.9% of participants who reported their preferred type of communication in our baseline demographic survey selected phone communication instead of email. If a participant did not confirm, we scheduled a different participant in their place.

Fig. 1

Covid-19 screening questionnaire.

In addition, we amplified our recruitment efforts by increasing in-person recruitment and the number of recruitment mailings sent each week. This adaptation was implemented in mid-August 2021. After implementation, we saw an average sign-up rate of 8.8 sign-ups per week starting from the month of September 2021 in comparison to an average of 3.0 sign-ups per week in July 2021. We also expanded our population's age range from 8 to 12 to 7–14 for participants who would otherwise have aged out due to the pandemic research suspension.

Increased rates of mail returns and delays

The pandemic also led to an increase in households moving, oftentimes out of larger cities like our study location in Boston [8]. This migration affected our ability to recruit participants via mailing, our primary recruitment method. Each week we sent out approximately 500 mailings with study and sign-up information; however, approximately 10% of the envelopes were returned by the United States Postal Service (USPS) due to an address change.

In addition, the USPS also experienced significant delays during the pandemic with an increase in mailed packages and employee absences [9]. These delays affected our recruitment delivery and equipment retrieval, thus resulting in delays to our study.

In addition to increasing the number of recruitment mailings sent out each week, we more frequently monitored the USPS tracking label on the self-addressed pre-stamped accelerometer packages that participants were expected to return after each visit, to ensure that participants were mailing their materials back in a prompt manner to compensate for potential added time in delivery.

Limited personnel inside the laboratory

The original protocol estimated 5–6 participants on each data collection day based on available personnel (four RAs, a wet lab supervisor, and a phlebotomist) and lab equipment. Family members other than primary parents/guardians were also allowed to stay in the lab to enhance accommodation. Once COVID-19 hit, University guidelines restricted the number of people allowed in the laboratory at a given time to a maximum of seven.

To adapt, the number of available slots for data collection each day was reduced from 5 to 6 to 2–3 to avoid overlap of participants and to account for one participant (1), additional family members (1) and lab personnel (5). Any additional family members or subsequently scheduled participants were accommodated in a separate space outside the lab.

Research team members contracting COVID-19/potential COVID-19 related loss

An additional challenge during the pandemic occurred when a research team member was exposed to COVID-19. After March 2020, university protocol instructed exposed individuals to self-isolate for 10–14 days and have two negative tests before being able to return to work. In January 2022, these restrictions changed to a minimum of five days of isolation. These precautions limited available staff for data collection and material preparation.

We enforced mask wearing, social distancing, and disinfection protocols to ameliorate this challenge. Before entering the lab space, RAs took participants' temperatures using a contactless thermometer and asked them about any COVID-19-related symptoms. Families were not accepted for the visit if any of the family members’ temperature was 100.4°F or said “yes” for having any COVID-19 related symptom or exposure [10]. Eligible families were asked to wash their hands and switch (or add) their face covering to the lab provided 3-ply disposable surgical masks along with face shields.

This COVID-19 screening was a double-edged sword, as more cancellations accumulated because of participants having symptoms that could also be attributed to the common cold or seasonal allergies. This occurred four times within January 2022 alone, resulting in 15% of all scheduled participants having to cancel their visit due to the child or close family member having had COVID-19 symptoms. To help this issue, we changed our protocol again: our study team purchased rapid tests to be taken at the beginning of each visit if the family believed the symptom the child participant was experiencing was not COVID-19 related. If the child tested negative, they could continue with their visit.

We have yet to see the impact of purchasing and using rapid tests. Since implementation in January 2022, only one participant requested the test since the parents believed the child to have seasonal allergy only. This perhaps could be due to the seven-day average of COVID-19 confirmed cases in Massachusetts steadily decreasing since the week of January 8th, 2022 [11]. Or, due to the commonwealth of Massachusetts purchasing 2.1 million at-home rapid antigen tests in December of 2021 which were distributed to over 100 municipalities with a larger proportion of families facing financial hardship, thus participants may have had their own rapid tests and no longer need us to give one to them [12].

Research assistant training

Before the pandemic, all research assistants went through an extensive in-person shadowing training process, which could no longer be accommodated due to the occupancy limit. This restriction added to the existing research team's workload and decreased training efficiency.

Most staff meetings and training were switched online to mitigate this challenge. In-person staff training was condensed and performed mostly on days that we did not collect data to avoid lab overpopulation. To compensate for the reduction of the shadowing training component, we implemented in-lab mock practice. The RA in training worked through multiple mock visits with a supervisor to get feedback and gain confidence before working with real participants. Each new RA was also scheduled to individually observe at least one full visit before conducting their own to retain some shadowing experience. Supervisors also frequently checked in with new RAs before and after their data collection visits to answer questions.

Discussion

Our aim in this study is to utilize technology to benefit children's health and increase their physical activity during the pandemic. As our team resumed our clinical trial after the initial COVID-19 lockdown, challenges abounded. We faced issues such as balancing COVID-19 screening enforcement and social distancing precautions with maintaining participant enrollment and lab efficiency. Thus, our team adapted, including increased reliance on various forms of communication and enhanced sanitization protocols. Most importantly, the protocol focus changed to how to be most accessible and safe for our participants while balancing participant recruitment, retention, and data collection.

A lesson learned while focusing on this participant-centric experience was that COVID-19 has a different impact on different communities. For example, within our sample, 47.9% of participants had incomes lower than $39,999. Demographically as well, 78.3% of participants identified their race as Black or African American and 41.7% identified their ethnicity as Hispanic, Latino or Mexican American. Recent COVID-19 studies have shown a significant inverse relationship between detected cases and median household incomes, as well as significant positive association between COVID-19 cases and the proportion of a population identifying as black [13]. Thus, some communities may be hit harder by the pandemic, leading to potential consequences such as higher dropout, cancellation, and rescheduling rates that other samples might not incur due to illness and its subsequent effects. In adopting a participant-centric focus, public health researchers may have better access to populations who are the most difficult to reach and who need the help the most. Facilitating better interactions may lead to better access to data and larger samples to improve statistical power.

The challenges and impact of COVID-19 may be long-lasting, especially for these underrepresented communities. We believe that some of our changes should be adopted more widely in the future even after the pandemic. For example, researchers should be more participant-centric by asking for preferred means of communication. Similarly, while staff training clearly suffered by reduced shadowing opportunities, using supervisors as “mock participants” helped significantly. By sharing practical tips from our experience conducting home-based physical activity clinical trials, we aim to inspire others to maximize potential public health benefits while also retaining clinical trial integrity with creativity and persistence. We hope our effort helps to provide some updated insight in research in media and technology, which has become an integral part of the pandemic life with significant implications for health and well-being for everyone.

Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.