MindKind: A mixed-methods protocol for the feasibility of global digital mental health studies in young people.

Background: While an estimated 14-20% of young adults experience mental health conditions worldwide, the best strategies for prevention and management are not fully understood. The ubiquity of smartphone use among young people makes them excellent candidates for collecting data about lived experiences and their relationships to mental health. However, not much is known about the factors affecting young peoples' willingness to share information about their mental health. Objective: We aim to understand the data governance and engagement strategies influencing young peoples' (aged 16-24) participation in app-based studies of mental health. We hypothesize that willingness to participate in research is influenced by involvement  in how their data is collected, shared, and used.
Methods: Here, we describe the MindKind Study, which employs mixed methods to understand the feasibility of global, smartphone-based studies of youth mental health. A pilot 12-week app-based substudy will query participants' willingness to engage with remote mental health studies. Participants will be randomized into one of four different data governance models designed to understand their preferences, as well as the acceptability of models that allow them more or less control over how their data are accessed and used. Enrolees will receive one of two different engagement strategies. A companion qualitative study will employ a deliberative democracy approach to examine the preferences, concerns and expectations of young people, with respect to remote mental health research. We also detail our engagement with young people as co-researchers in this study. This pilot study is being conducted in India, South Africa and the United Kingdom. Conclusions: This study is expected to generate new insights into the feasibility of, and best practices for, remote smartphone-based studies of mental health in youth and represents an important step toward understanding which approaches could help people better manage their mental health. Copyright:

Introduction

For adolescents globally, mental health conditions constitute a major burden of disease, with one out of seven adolescents estimated to be affected in 2019 . According to the World Health Organization, up to 20% of children and adolescents worldwide experience one or more mental health conditions with three-quarters of these illnesses developing by their mid-20s . The burden has been exacerbated by the current Sars-Cov-2 pandemic, during which anxiety and depression have been particularly prominent . Despite the high prevalence rate, strategies for prevention and approaches to treating anxiety and depression in young people remain limited due to a myriad of factors, such as variable life circumstances and lived experience, lack of resources, shortage of trained health care providers, and associated social stigma and discrimination.

To understand what kind of approaches will work for whom and why, it is important to gather data about the lived experience of young people with anxiety or depression. In order to study the variable ramifications of “lived experience” on disease trajectories and management, longitudinal data on lived experience must be collected across large study cohorts to detail the many aspects of an individual’s life, that may relate to or influence their mental health status, such as sleep, physical activity, social relationships, etc. Such data can be collected directly from individuals using digital technologies like smartphones. The ubiquity of smartphones and other modern technologies provides an opportunity for remote data collection of mental health variables in a convenient and non-intrusive manner. For longitudinal data especially, it can ensure rapid data collection with ease, in real life settings, without travel or administrative barriers . Additionally, it opens avenues for large-scale data collection across borders that can yield data rich in the variability that could be key in understanding mental health.

The use of smartphones to capture participant-reported outcomes or e-diaries has become commonplace in health and mental health research . In addition, mobile apps and digital sensors have been used in a number of research studies to monitor symptom variability . Along with administering surveys, smartphone-based applications also enable the collection of data using evidence-based scales and instruments on factors that impact youth mental health along with passive data, such as the number of steps taken or screen time , thus making them an effective medium to better understand young people's social, physical and emotional lives. However, the collection, processing, and use of such data, which is considered highly personal by many individuals, will need to be sensitive to the interests of the young people contributing their data. Very little is known about the factors influencing young peoples’ willingness to participate in such research, and across many domains, participation in remote studies tends to drop off quickly , so understanding the factors that attract and maintain participation are important for the success of such research.

Here we describe the protocol for MindKind, a mixed-methods study to understand the feasibility of developing a global mental health database of digital data collected from young adults using smartphones. We hypothesize that young peoples’ willingness to participate in digital research is influenced by their ability to be involved in how their data is collected, shared, and used. To that end, we will directly test a range of data governance models that promote participant-led open data practices and two different app-based engagement strategies to identify those that lead to the highest participation. Our two-pronged approach pairs a pilot (quantitative) smartphone-based study with a qualitative study to understand the participatory behaviours, concerns, and desires of young people with respect to mobile mental health research.

This study is being conducted in three countries: India, South Africa and the United Kingdom (UK), which were chosen for their range of economic, socio-cultural and regulatory landscapes. Throughout the project we centre our work on the voices of young people, by grounding our work in a youth-adult participatory research approach whereby we seek to involve youth stakeholders and researchers as equal partners , including as youth leads employed as members of the study staff, youth panel members, and as research participants. We seek to engage young adults with an interest in mental health as co-researchers and partners throughout the entire process, from design, prototyping, and testing of the study frameworks.

Methods

Ethics

MindKind was approved by the relevant Institutional Review Boards and Ethics Boards in the U.S. (WIRB #20212067), UK (University of Cambridge - Department of Psychology Research Ethics Committee: Ref. PRE.2021.031 and University of Oxford: Ref. R73366/RE00), South Africa (Walter Sisulu University #029/2021 and the Department of Higher Education and Training), India (India Law Society #ILS/242/2021), and by the Health Ministry Screening Committee (HMSC) in India.

Study overview

MindKind is a feasibility study being conducted in India, South Africa, and the UK to capture the preferences and perspectives of youth from a set of diverse cultural backgrounds. The mental health data collected are not intended for analysis, but rather to understand the degree to which young adults are willing to share this information. The study’s aims are underpinned by the assumption that youth involvement in such a databank is essential to its success . In addition to the study design, we describe the roles of Youth Advisors and our Data Use Advisory Group ( Table 1) in the development process. The study will employ mixed methods and consists of two substudies ( Table 2).

Table 1.

MindKind advisory roles.

	Role or Panel Name	Composition	Time Commitment	Age	Other Qualifications	Role
Youth Roles	Professional youth advisors (PYA)	1 per research site	Full time	18-24	· Lived experience with mental health challenges · Prior experience working on mental health-related, adolescent-focused, or community engagement projects and programs	· Core Research team member · Help guide decisions and research directions · Liaise between the country-specific YPAGs and the project teams · Lead recruitment for their respective YPAG · Facilitate YPAG meetings, synthesizing panel feedback · Provide recommendations to other project teams, and present their findings regularly at Steering Committee meetings
	Youth panel advisory groups (YPAGs)	12-16 members per country-specific panel	Twice monthly meetings (1-2 hours each) + Ad hoc asynchronous assignments	16–24 (UK); 18–24 (India, South Africa)	· Lived experience with mental health challenges · Access to a phone or computer to participate in panel meetings	· Provide feedback on key aspects of the study design and data collection · Participate in monthly meetings and asynchronous virtual chats (e.g., via WhatsApp) · Participate in small group projects and feedback rounds in between meetings (as needed)
	International youth panel (IYP)	9-15 members (3-5 per YPAG)	Once monthly (1 hour each)	16–24 (UK); 18–24 (India, South Africa)	· Selected democratically by each YPAG	· Cross-site networking · Consensus building for key decisions · Complete regular trainings on research capacity building
	Global youth panel (GYP)	15 members from each of the 3 site countries + other high-, and middle-income countries (e.g., USA, Canada, Kenya, Nigeria)	Once monthly (1 hour each)	18-30	· Selected by the academic study team based on their past experience on youth panels and in advocacy groups for mental health issues among young people	· Provide high-level feedback on project decisions that could inform future testing and rollout of the MindKind study beyond our current three study locations
Researcher Roles	Data Use Advisory Group (DUAG)	18 researchers from seven countries (Australia, Brazil, India, Nigeria, South Africa, United Kingdom, and United States)	2-4 times per year + ad hoc engagement (e.g., follow up emails and surveys)	NA	Background in/with · Open science and data sharing · Wellcome ‘active ingredients’ commission 17 and/or · Adolescent mental health research, social-medical sciences, and/or clinical practice	· Provide perspectives on scientific uses for a global mental health databank, research ethics, governance models, data storage and accessibility, data use agreements, and researcher qualifications

Table 2.

Overview of MindKind research questions and youth and DUAG involvement.

Research Question	Quantitative substudy methods	Qualitative substudy methods	Youth participation		Data Use Advisory Group involvement
			Youth Leads	Youth Panels
What are youths’ preferences for how researchers access their data?	Participants randomized to governance Group A ( Figure 1) choose their preference.	Facilitated discussion within deliberative democracy sessions of different data governance options.	Led youth panel 3+ rounds of discussions regarding governance model options. For the quant. substudy, the topics of contention were selected for formal testing. In the qual. study, PYAs helped refine the governance model options and convey these options in plain language. PYAs provided granular, ongoing feedback in developing the governance options. Assisted in the adaptation of the educational materials for the qual. subtudy and recorded audio narration. PYAs will facilitate the deliberative democracy sessions and will help develop the analytical framework and analyze data.	Participated in 3+ rounds of feedback collection regarding governance model options in the quantitative and qualitative substudies. For the quant. substudy, the topics of contention were selected for formal testing. In the qual. substudy, they helped refine the governance model options, convey these options in plain language, and served as mock participants. Will review and provide feedback on the analytical findings of the study upon study completion.	Contributed to scoping and defining the governance model options. In the qual. substudy, DUAG members’ feedback contributed to the advantages and disadvantages of governance models presented in the substudies. Provided input on equity of data access, particularly for less resourced settings.
What are youths’ preferences for who controls access to their data?	Participants randomized to governance Group A ( Figure 1) choose their preference.	Direct inquiry within deliberative democracy sessions.
Does data governance model affect enrollment or engagement with the app?	Comparison of enrollment rate and engagement for participants randomized to governance Groups B, C & D ( Figure 1).	NA
Identify the consensus data governance model(s) for an open yet privacy-preserving global mental health databank, from the perspective of multinational young people.	NA	Consensus-building deliberative democracy sessions.	Facilitate qualitative data collection around the research question.	Contribute to early scoping on which data governance model(s) would be presented.	Contributed perspectives on preferred data governance models during study design phase.
Understand the concerns, hopes, and expectations of multinational youth for such a databank with regards to the return of value to youth participants and youth participation in databank governance.	NA	Facilitated discussion within deliberative democracy sessions of different data governance options.	Facilitate deliberative democracy sessions. Review prototypes of a global mental health databank and provide feedback on options for website and app design for final recommendations to Wellcome Trust.	Review prototypes of a global mental health databank and provide feedback on options for website and app design for final recommendations to Wellcome Trust.	Review prototypes of a global mental health databank and provide feedback on options for website and app design for researcher perspectives.
Do participants engage more with app-based research when they have a choice about the topics covered?	Comparison of engagement for participants randomized to engagement Arm 1 and Arm 2 ( Figure 2).	NA	Led a series of ~4 youth panel meetings on survey design and engagement strategies for young people participating in the study. Directly impacted the selection of active ingredients topics. Provided feedback on aspects of study design, engagement approaches, and active ingredient measurement to contribute to study design. Also, worked closely with site teams on the development of recruitment materials. Following study completion, will review study findings to discuss key messages and dissemination approaches.	Participated in ~4 panel meetings on survey design, engagement strategies for young people participating in the study. Advised on the selection of active ingredients topics. Provided input to their PYAs on the development of recruitment and retention materials. Following study completion, will review study findings to discuss key messages and dissemination approaches.	NA

The quantitative substudy will pilot an app-based protocol, which queries participants’ data governance preferences as well as the relative acceptability of governance models. The aim is to understand how study participants would prefer their data governed and accessed and how governance policies affect study participation. As a secondary aim, we will test two different engagement strategies to determine how short- and long-term engagement may be impacted by offering participants a choice of which mental health mediators--‘active ingredients’-- they contribute to the databank. Here we define ‘active ingredients’ (AIs) as factors which have been shown to influence mental health .

We focus on four AIs: sleep, body movement/exercise, social connections, and positive activities. In addition to collecting information on these AIs, the quantitative substudy will also gather demographic information and mental health instruments in order to understand the willingness of participants to share information about their mental health on an ongoing basis. Participants will interact with a custom Android app to enable rapid and convenient data collection. Through a purpose-build app, we will examine such factors as consent to enrol, the types of data contributed, and duration of data contribution.

The qualitative substudy will collect data from deliberative democracy sessions. Deliberative democracy is a method that joins communities in discussions of complex ethical issues by providing education to inform discussion and engaging participants in dialogue leading to iterative consensus building . We apply the method to youth databank governance preferences. We aim to (1) identify the consensus data governance model(s) for an open yet privacy-preserving global mental health databank, from the perspective of multinational young people; and (2) understand the concerns, hopes, and expectations of multinational youth for such a databank with regards to the return of value to youth participants and youth participation in databank governance ( Table 2).

Interplay between quantitative and qualitative substudies

Both substudies inform our understanding of how to develop a global mental health databank for young people ( Table 2). The quantitative study asks questions about databank engagement, preferences, and acceptability within certain constraints. The qualitative study exposes participants to options for data management and storage that are not feasible to ask of participants in the quantitative study. Whereas the quantitative study collects data that researchers can compare across regions, the qualitative study puts participants from different countries in direct dialogue with one another through multi-national deliberative sessions. This multinational session data will offer insights into why certain preferential differences may arise in the quantitative app-mediated study. The qualitative study generates data that can be implemented in future iterations of the app-based (quantitative) study as it asks about an ideal scenario to aim for. The two substudies will run concurrently, and results from the two will be compared to understand participants’ ideals and their effect on participation relative to current research standards.

Community engagement of Youth and Researchers

Youth engagement. This study leverages community-based participatory research approaches to guide the engagement of young people with lived experience of mental health challenges ( Table 1 & Table 2). We define “lived experience” as self-reported experience with mental health challenges that cause a significant change in day-to-day functioning and contribution to community .

By including young people as co-researchers and data contribution partners, we aim to: 1) centre our work from a social justice and health equity lens; 2) improve the quality and appropriateness of our study design, data collection, and data analysis; 3) empower young people as active agents for change in their communities; and 4) build capacity among young people around mental health and mental health research .

Feedback from youth panels informs and guides project decision making as much as possible ( Table 2). In addition, all youth recommendations and decisions are documented and will be revisited for future, larger-scale studies based on this work. Additionally, professional youth advisors (PYAs) provide regular feedback on their roles in the project to the academic study team. This allows for meaningful review and revision of youth role descriptions and deliverables and will help us identify opportunities to improve equitable collaboration and devise study-specific metrics of successful youth engagement and study co-production.

Researcher engagement. On a smaller scale, we also engage stakeholders across research areas pertinent to our study (e.g. open science, ethical data sharing, active ingredient research, adolescent mental health, sociocultural mental health, etc.). This Data Use Advisory Group (DUAG) provides regular guidance and feedback about the MindKind Study and future databank ( Table 1 & Table 2).

MindKind study design

To participate in the MindKind study, youth participants must live in one of the participating countries and be legally able to provide consent (age 16–24 in the UK or 18–24 for India and South Africa). Youth are eligible if they can follow study instructions, read and understand English, and have access to an Android mobile phone (for quantitative substudy participation only).

The quantitative substudy is designed to assess the preference and acceptability of different data governance models and their effect on engagement patterns over time. Secondarily, it is designed to test whether choice of study topic affects study engagement ( Table 2).

Study design: governance models

Given the sensitivity of mental health data, we are seeking to understand whether prospective participants of a future global mental health databank have a preference for data governance models which give participants more control over who can access the data and for what purpose. We will also seek to understand whether these preferences impact enrolment. In order to assess these questions, youth participants will be randomized, in equal proportions, to one of four different governance experiences ( Figure 1), the first of which (Group A) assesses preference, with the remaining three (Groups B, C, D) addressing acceptability of various data governance models.

Figure 1.

Governance study design.

Potential quantitative substudy participants are randomly assigned to one of four consent models. Group A is designed to assess what practices are preferable to study participants. Groups B-D are designed to assess the acceptability of current standards relative to models that allow participants a greater voice and more data security.

In order to assess the preference of young people with regards to data governance, participants randomized to Group A will be prompted to select how researchers are allowed to access their data, and who controls access to the data. The selection of these options was informed by the disparate preferences expressed by youth co-researchers versus the DUAG.

Choice 1: How should researchers be allowed to access the study data?

Researchers should be allowed to download a copy.

Researchers should only be allowed to access the data in a secure server.

Researchers should only be allowed to see a recreated data set, not the real data. If researchers want to study the real data set, they have to ask the data steward to run their analysis for them. The steward only gives the researcher back the result, not the data.

Choice 2: Who controls the data?

Democracy: study participants who select this option get to vote on how the data is used, and the most popular terms are applied to all data regardless of how an individual votes. The results of the vote are shared with participants before data are shared. Any participant who disagrees with the vote may withdraw from the study. See Appendix 1 ( Extended data) for voting questions.

Volunteer community review panel: participants selecting this option may choose to volunteer to serve as a data use request reviewer, taking one-week turns in this role on a rotating basis. Researchers will submit a statement telling the reviewers why they want to use the data. The reviewers will apply a set of criteria to decide yes or no. These criteria will be determined in advance by the whole group of volunteer reviewers.

Professional review panel: a paid panel will review data requests. This panel is a group of participants paid by the funder of the databank and may include research professionals (e.g., research ethics professionals). As above, researchers will have to submit a statement telling the reviewers why they want to use the data. The reviewers would decide yes or no, based on a set of criteria to which will be developed in advance by the group.

Participants randomized to Group A will be asked to select their data governance choices prior to consenting. Additionally, participants who select ‘Democracy’ for choice two will be asked to provide their preference on four questions about data terms of use (Appendix 1 , Extended data), which constitutes their democratic vote ( Figure 1).

In order to assess the acceptability of current governance standards relative to those that give participants a greater voice regarding how data are accessed and used, participants randomized to Groups B, C, or D will be presented with a pre-specified governance model. These three ‘acceptability’ experiences were selected by the research team to test (1) whether democratic determination of data terms improves enrolment over current researcher-driven norms, and (2) whether limiting data access to a restricted server further improves enrolment. Specifically, the three models are:

Research norm (Group B): This option presents current researcher community norms for data use, whereby researchers will be able to download a copy of the data from the databank following strict data security rules. Data may be used, unrestricted, by both commercial and non-commercial researchers.

Youth informed democracy with download (Group C): Study participants will vote as described in the Democracy Choice above. Under this model, researchers are allowed to download a copy of the data.

Youth informed democracy without download (Group D): Study participants will vote as described in the Democracy Choice above. Under this data governance model, data may only be accessed via a restricted server.

In order to mirror the experience in a typical study, participants will be exposed to an informed consent specific to their data governance model and can choose to either join or not. They will have no exposure to other potential governance models. In quantifying the difference in enrolment rates between participants in each group, we can assess whether democratic determination of access terms improves enrolment (Group C vs Group B), and whether restricting data download additionally improves enrolment (Group D vs Group C). We will also assess the effect of governance models on study engagement and retention (See the Data Analysis Plan Engagement section for details).

Study design: engagement

Following enrolment in the study, participants will use the study app to complete daily activities and surveys for the course of the 12-week study ( Figure 2). The study poses questions about four active ingredients (AIs) which have been shown to influence mental health: sleep, body movement/exercise, social connections, and positive activities . Participants will focus on one of these domains in four-week rotations. For example, a participant may receive questions about body movement for weeks one to four, positive activities for weeks five to eight, and social connections for weeks nine to 12. See the “ Surveys and data collection” section for more details on the content of these domain surveys.

Figure 2.

MindKind app study design.

The 12-week study is composed of three, four-week rotations focusing on four active ingredients influencing mental health (sleep, body movement, social connections and positive experiences). Participants will be randomized into one of two arms: the first of which allows participants to select their AIs of focus and the second of which randomly assigns the AIs to which participants are exposed. In this example, the ARM 1 participant has selected the topics Body movement, Positive activities and Social connections for their topics, while the ARM 2 participant has been randomly assigned to Sleep, Positive activities and Body movement for their 3 AI topics.

In order to understand whether the choice of survey domain impacts a participant’s short- or long-term engagement in the study, we will independently randomize participants into two different arms (in equal proportion). Participants in the first arm select their AI topic at the beginning of weeks one, five, and nine. Participants in the second arm are randomly assigned to their AIs for each of the three, four-week rotations ( Figure 2).

Recruitment

The quantitative substudy aims to recruit 4500 young people (1500 from each country) to download the study app, with a minimum of 10% of the 4500 young people recruited having lived experience of anxiety and depression. The sample size was selected based on expected engagement in order to achieve a program goal of 100 participants per country completing the 12-week study.

Rolling recruitment will begin in the third quarter of 2021. Youth will be recruited in the following ways:

Through social media given limitations on in-person activities during the ongoing COVID-19 pandemic. Posts and advertisements will be placed on popular youth accessible social media platforms such as Instagram, Twitter, Facebook, LinkedIn, Reddit, and WhatsApp to reach young people in each of the three study countries.

To ensure that data collection is representative of a broad population in each country, the three sites will collaborate with intersectoral organisations (organisations working on sexual and reproductive health research, disabilities, sexuality, race/caste etc) with the help of emailers and posts.

Each of the sites will also reach out to partner (youth) organisations and spread the word within their networks along with using existing researcher contacts.

We will also recruit students at educational institutions (e.g., schools, colleges and universities) with the help of posters/flyers, and existing contacts between the site team and identified institutions.

Networks of young people will be tapped into by PYA and YPAG participants for snowball recruitment.

Recruitment materials such as social media posts, posters, flyers, videos etc, will be developed in English for all the three sites. Additionally, to reach a more diverse population, recruitment materials will be translated to other languages in South Africa (IsiXhosa and seSotho) and in India (Hindi, Marathi and Tamil). However, the study app will be available only in English.

Youth co-researchers have been heavily involved in the development of recruitment plans, providing feedback on the recruitment materials and helping disseminate the final materials to ensure that all recruitment materials are youth friendly and culturally appropriate.

Due to concerns about cellular data costs in South Africa, study participants in this country will be given a small stipend to subsidize their data plans in order to facilitate participation. Stipends of R150 will be paid per four weeks of participation in the quantitative substudy. Similarly, a small stipend pool exists for participants in India who find it difficult to pay for data. However, given the relatively low expense for cellular data in India , the need is not expected to be widespread. No stipends are offered for participants of the quantitative substudy in the UK.

Data collection

On enrolment, a baseline survey is administered to catalogue the participant’s background and experiences with mental health. For ease of administration, these are divided into four sections. ‘About you’ includes the topics demographics and socio-economic status (Appendix 3 , Extended data). ‘Your environment’ includes food security (USDA Food insecurity survey (six-item)) , neighbourhood safety and cohesion questions (PhenX Neighbourhood Safety, and PhenX Collective Efficacy) and questions related to history of exposure to violence (Appendix 3 , Extended data). ‘Your habits’ includes questions pertaining to hobbies, physical activity and phone use habits (Appendix 3 , Extended data). ‘Your health’ includes questions pertaining to physical ability (WHODAS 2.0 (12-item) ), depression (PHQ-9 ), anxiety (GAD-7 ), and history and management of mental health (Appendix 3 , Extended data). These baseline questionnaires are administered on study day zero ( Figure 3). On the following day, participants begin their first AI-rotation for weeks one to four. The second and third rotations occur weeks five to eight and nine to 12, respectively ( Figure 2).

Figure 3.

Baseline, daily, and weekly surveys.

MindKind is a 12-week study consisting of a baseline survey followed by four-week rotations focusing on a single “active ingredient” (AI). On the seventh day of the week, a long survey is administered consisting of a standard instrument pertaining to the topic of the AI, as well as PHQ-9 and GAD-7. Preliminary testing showed that this weekly survey took 9 minutes on average (range 3 to 22 min). During the remaining days participants receive short questionnaires including a standard mood question and three to five AI-specific questions. Preliminary testing indicated that the daily surveys took no more than 5 minutes to complete (range 30 sec to 5 min). Participants are prompted to journal on one of those days. At the beginning of weeks five and nine, a new AI is presented (Arm 2) or selected by the participant (Arm 1). This design was selected based on feedback from the PYAs and YPAGs to minimize burden to participants while still collecting rich longitudinal data. PYAs and YPAGs were also consulted in the selection of the specific AIs chosen for implementation.

Surveys and active data collection: The questions posed to participants throughout the study focus on the interplay between mood and four different AIs (sleep, social connections, body movement or physical activity, and positive activities) ( Figure 2– Figure 3). Study participants focus on one AI at a time, in four-week rotations. On days one to six of the week, participants are asked a standard mood question:

Pick the response that describes how you felt today:

Worst ever

Bad mood

Average

Good mood

Best ever

along with a short (three to five item) AI-specific questionnaire (Appendix 2 , Extended data). They are also prompted to journal on an AI-specific or general topic on one of those days. On the seventh day, participants are asked to complete a long survey related to their AI topic (Insomnia Severity Index (seven-item) , UCLA Loneliness Scale (three-item) , International Physical Activity Questionnaire (seven-item) , Behavioural Activation for Depression Scale for the sleep, social connections, body movement and positive activities AIs, respectively), as well as PHQ-9 and GAD-7 to get a deeper understanding of their mood ( Figure 3).

Passive data collection: Participants can opt in to provide passive data collection about their phone activity and environment in order to understand their phone use habits. The statistics collected are daily screen time (a daily log of when the phone’s screen is unlocked), charging time (a log of when the phone is connected to its charger), battery statistics (a log of battery charge throughout the day via the Android ACTION_BATTERY_CHANGED call), data usage (hourly reporting of amount of data transmitted and received), ambient light as measured by the phone’s light sensor (sampled for ten seconds every five minutes in order to minimize battery consumption). No information will be collected that could violate a participant’s privacy, such as information about specific activities or apps used on the phone, call logs, or the content of messages. No identifiable location data will be collected.

Technical implementation: The study will be implemented in two separate pieces of technology. Eligibility checking, account creation and enrolment occurs via a website which has been optimized for mobile device experiences. Once a participant has enrolled, they are prompted to download the MindKind Study App from the Google Play store. The app requires an Android phone running Android 5.0 or higher. Participants use their phone number as a mechanism to create an account on the enrolment website, and to login to their account on the study app. Each time they request to login, a unique code sent to their phone via SMS validates their login. Due to SMS messaging restrictions, participants located in India created a 6-digit PIN in lieu of SMS validation for login.

Both applications will store data in the Bridge Server, a set of web services developed and operated by Sage Bionetworks. Bridge exposes a REST style web services API designed to allow collection and management of mobile health data from a variety of apps. This service has been used by Sage and other research organizations to support a variety of health studies on both the Android and iOS platforms. Bridge provides apps the ability to securely create accounts for participants, and record consent and other personal information separately from study data intended to be shared with research teams.

Data management: Data from both the enrolment website and study app will be sent to Sage Bionetworks’ Bridge Server as json files. Coded study data, consisting of survey responses and passive data measurements, will be exported to Sage’s data sharing service, Synapse, for access by the study team . Synapse is a general-purpose data and analysis sharing service where members can work collaboratively, analyse data, share insights and have attributions and provenance of those insights to share with others. Synapse is developed and operated by Sage Bionetworks as a service to the health research community. Programmatic access to Synapse via R and Python Clients allows us to curate the json files from the surveys into tabular formats by survey type in order to allow for analysis.

Data protection: The data collected through the phone app will be encrypted on the participant’s phone. The app will not access other applications or user contacts, photos, texts, or emails on the participant’s phone. Identifiable account information will be stored separately from participants’ study data, and the encrypted mapping between personally identifiable information and the de-identified study data is only accessible by key personnel with controlled access. The de-identified coded study data will be stored on Synapse as described above.

We maintain strict information technology procedures to safeguard information and prevent improper access. Our Information Security Program is based on the ISO 27000 series of Information Security Management System standards and is supported by special NIST publications. The Bridge server and Synapse research environment run on Amazon Web Services (AWS) S3 Cloud-based infrastructure. It is protected by data privacy and security controls standards in healthcare information technology, including the provisions required by HIPAA/HITECH and GDPR. Files and database storage supporting the application are encrypted with AES-256. Transmissions through Synapse interfaces are encrypted with SSL/TLS, enforced by technical policy. Synapse operates under an IRB-approved governance process that includes well-documented Terms and Conditions of Use, guidelines and operating procedures for handling data, data security measures with strict information and privacy-enhancing technologies, as well as the right of audit and external reviews. These end-to-end security and end-to-end privacy measures safeguard the confidentiality and integrity of the data.

Data analysis plan

All analyses described below will be performed within-site as well as across-site, adjusting for site. A sensitivity analysis will be performed, excluding South Africa, due to the potential confounding introduced by participant payment.

Governance preference: This analysis leverages the Group A ( Figure 1) selections for Option 1 and Option 2 to understand the degree to which study participants have a preference about how their data are accessed, both on who determines which investigators can access the data, and by what means they access the data. Outcome variables for this aspect of the substudy are the prospective participant’s (1) model choice for governing standard for the data and (2) model choice for researcher access to the data. The primary analysis will be a summarization of participant preference for each of the two questions. A chi-square test will be used to assess the statistical significance of the differences observed between the three options. A secondary set of analyses will assess the degree to which age, gender, and self-reported present or past mental health issues are associated with governance model preference, both globally and within-site. This analysis will also be performed with a multinomial logistic regression adjusting for these three variables.

Governance acceptability: This analysis leverages the enrolment rate for Groups B-D to understand the degree to which a governance model that improves usability and access by investigators, affects study enrolment over models that give participants more control (i.e. democratic determination of data terms) and offer better data security (i.e. secure server access only). The outcome variable for this aspect of the substudy is whether or not the prospective participant enrolled (yes/no). A logistic regression to test the effect of the governance model will be fit, adjusting for age, gender, self-reported present or past mental health issues, and site (global model only). The primary comparisons of interest are Group C vs Group B (democracy vs current norms) and Group D vs Group C (server access vs data download). A secondary analysis will test an interaction term between the model and self-reported mental health issues.

Engagement: The goal of this portion of the study is to understand participant engagement and compliance with the study protocol during the course of the 12-week study. We will measure participant engagement based on the following metrics: 1) study completion rates, i.e. proportion of participants that remain active in the study through 12 weeks as well as interim time points (e.g. four and eight weeks); 2) types of data (surveys and passive data streams) shared by participants and for how long; 3) total active tasks completed; 4) time to dropout (i.e. time point of the last task completed).

For study completion and interim participation, the binary outcome variables ‘participation at time X’ (yes/no) can be modelled using logistic regression to test the effect of the engagement model, adjusting for various socio-demographic factors (e.g. age, gender, self-reported history of mental health challenges), governance model (self-choice, or specific model presented) and site (global model only). For total task completion, the number of tasks completed by a participant will be modelled using Poisson Regression using the same independent variables as previously stated. For assessing the types of data shared by participants, we will compute the aggregate statistics on the number of unique data types shared by participants and for how long. Using a logistic regression model, we will assess the potential differences in data sharing aggregate statistics. Time to dropout can be analysed using a proportional hazards model with censoring at the study end. For each of these three analyses, we will also secondarily examine the effect of the governance model on engagement.

Study design

The qualitative substudy aims to (1) identify the consensus data governance model(s) for an open yet privacy-preserving global mental health databank, from the perspective of multinational young people; and (2) understand the concerns, hopes, and expectations of multinational youth for such a databank with regards to the return of value to youth participants and youth participation in databank governance ( Table 2). We will employ a deliberative democracy approach for this study . Deliberative democracy is a method for community engagement in the complex ethical issues surrounding emerging technologies for which most people have not yet formed strong opinions. Deliberative democracy is a distinct qualitative research approach, differing from focus groups by the purposeful provision of educational intervention to enrich group discussion. Further, there is an emphasis on discussion leading to iterative revision of opinions as participants integrate new information and others’ perspectives.

We will conduct two rounds of deliberative democracy sessions. In the first round, up to ten cohorts of five to seven participants will be convened per country (approximately 150 total participants). A sample size of n=50 per site is characteristic of deliberative democracy studies . The individual group size of five to seven participants is characteristic of a typical small group in in-person studies . Each will explore data governance models and voice their concerns and hopes regarding data governance for a global mental health databank for youth, identifying points of consensus and disagreement. Standard educational materials will be co-created and disseminated prior to the deliberative sessions to prepare participants. These materials may be disseminated as written or in other multi-media formats (e.g., audio, video). Participants will have the opportunity to ask the research team any questions prior to the sessions. Cohorts will meet virtually for up to two hours.

In the second round of deliberation, we will convene up to ten multinational cohorts; all participants in the multinational sessions will be drawn from the first round. Again, cohorts will meet for up to two hours in two 60-minute sessions. Each cohort includes six to eight participants: two to three participants from each country (60 total participants).

This arrangement of sessions allows participants to: 1) develop expertise over time both with content and with the deliberative approach; 2) appreciate the similarities and differences between country-specific and multinational perspectives on data governance; 3) reflect on the intersection of these topics with their own lived experience.

The quality of deliberative democracy sessions will be evaluated using criteria proposed by DeVries et al., 2010 , including equal participation of all participants, respect for the opinions of others, willingness to adopt a social (rather than individualistic) perspective, and reasoned justification for one’s positions. These data will be gathered by the research team participant exit surveys.

Co-creation

YPAGs and the IYP engaged in co-design with the academic study team on the following topics: a) ensuring representation from marginalized and vulnerable groups in recruitment and participation; b) developing study materials including: informed consent, educational materials, and facilitator guides; c) identifying the challenges of virtual discussions and developing mitigation strategies to address equal engagement and power imbalances; d) planning for involving youth co-researchers in data analysis; e) dissemination of and access to study findings in a way that is engaging and inclusive of the people who have participated.

Recruitment

Participants will be recruited directly through YPAG members and existing volunteer rosters, existing researcher contacts, and snowball recruitment through these contacts. Further, social media (passive, active), direct email advertising, and posters may be used. Additionally, a single-time pop-up message in the MindKind app will recruit from young people participating in the quantitative substudy with a goal that half have previous exposure to the study. Young people with lived experience of anxiety and/or depression will be preferentially enrolled.

Deliberative sessions will be stratified by age and co-enrolment or lack of co-enrolment in the quantitative substudy. Youth will participate in sessions with others of similar age. For single country sessions, there will be up to three age cohorts (e.g., 16- to 17-year-olds in the UK; 18- to 20-year-olds in all countries; 21–24-year-olds in all countries). For multinational sessions, there will be at least two age cohorts (e.g., 16- to 20-year- olds; 21- to 24-year-olds). In single country sessions, we will aim for an equal distribution of participants across these two variables (i.e., age and co-enrolment in the quantitative substudy), but will have no fewer than one-third in each category, except for the UK age groups, which will have no fewer than one quarter of the total UK study population enrolled in any one age category. Additionally, we will seek a sample from a diversity of backgrounds in relation to gender, geographic location, socioeconomic status, educational attainment, ethnicity, religion, and first language. We will continue sampling participants until we have recruited a diverse set of individuals.

Given the amount of time required for pre-learning activities and participation, qualitative substudy participants will receive a modest incentive for participation in the form of a gift card (UK, India) or airtime voucher (South Africa).

Data collection and analysis

There are two primary outputs of deliberative democracy processes: deliberative output and analytical output . Deliberative outputs are statements of consensus or disagreement that arise directly from discussants. These statements will be captured by the facilitator and will be ratified by discussants themselves before the end of their cohort session. We will employ the framework method to capture analytical outputs regarding concerns, hopes, and expectations of discussants on return of value for participation in such a databank and the concerns, hopes, and expectations of discussants regarding youth participation. The framework approach is particularly useful in multi-disciplinary research teams that include lay people with less qualitative data analysis experience.

Discussion session data will be transcribed verbatim, de-identified, and checked for quality against audio recordings by the academic research team. De-identified transcripts will be uploaded to a cloud-based qualitative data analysis platform (NVivo). The academic research team, in collaboration with youth researchers, will code each transcript on this shared platform. The study team will identify a priori codes based on study goals, and emergent codes will be developed iteratively during the coding process. Coding discrepancies will be resolved through whole group discussion. Youth co-researchers will collaborate with the academic research team in the analysis process. All analytical outputs will be reviewed and ratified by the youth co-researchers and academic research team collectively.

Data protection

The deliberative democracy sessions will be recorded, transcribed, and directly identifying information will be removed. Pseudonymized transcripts and notes will be shared among the research team only via the US cloud-based GDPR-compliant qualitative coding software platform.

Discussion

The goal of this feasibility study is to further our understanding of the potential issues and challenges with developing a databank from remote, smartphone-based assessments of mental health in youth. Our findings will then inform the development of a larger global mental health databank and any future evaluations. Both the quantitative and qualitative study in the UK and South Africa are due to begin imminently, with the Indian site to follow once governmental approval is received.

We believe that the nature and extent of youth involvement in shaping an ambitious global mental health databank is uncommon in current research practices. The involvement of professional youth advisors in each of the sites, who in turn establish and run young people advisory groups (YPAGs) has been critical in ensuring that the study design, methods, engagement strategies, and app design are tailored to young people. While the development of the professional youth advisor role requires capacity building that must be undertaken for them to meaningfully engage with the study, the benefits of involving young people directly are manifold. To the best of our knowledge, this is the first time multi-site professional youth advisors have been engaged quite so closely in shaping the course of a project aimed at improving adolescent mental health globally.

Challenges

This study benefits from site teams working in different contexts and is unusual in that two of the three study sites are from low-to-middle income countries (LMICs). However, these differences present some challenges. The sites have had to contextualize the data safety management and storage policies as per the sites and country-level laws and policies. In a digital feasibility study, the sites have had to ensure that concerns regarding data transfer to the main study site, privacy, and implications on the use of data collected were fully considered as a consortium along with the youth leads. Consistent, on-going consortium level meetings, consultations from experts on ethics, law and mental health researchers were some of the processes taken to ensure that the consortium is responsive to the ethical and cultural requirements.

This project has also faced planning and design challenges imposed by the COVID-19 pandemic. Many efforts initially envisioned as in-person, principally the youth panels, were transitioned to virtual venues. This has led to concerns about representation from marginalised and vulnerable groups, impacted by the inability to own a device to access the internet, gendered use of technology, and language barriers since the meetings were primarily held in English.

Integrating voices of young people also required strategies for protection against harm. This resulted in each site creating a safeguarding protocol that was contextually relevant for the group. Safeguarding included taking all reasonable steps to prevent any form of harm, abuse or neglect from occurring; protecting people's health, wellbeing and human rights; and further, taking reasonable steps to respond appropriately when harm or abuse does occur. As part of the process, the study was co-designed with young people and the consent forms and study material explained the purpose of the study in a simple manner.

We anticipate future challenges in implementing this study. For example, differences in ethical and institution specific procedural requirements that could potentially lead to delayed start dates for some study sites, which has implications on meeting the project timelines and milestones. Moreover, the varied timeline across sites has implications on equal participation from the study partners and youth engagement, such as formation of the youth panels or recruitment of the study team members across sites.

We also anticipate that recruiting 1500 young people for the quantitative study in the three sites may be challenging due to the size of this group proposed to be recruited and specific constraints in contexts. For example, there are lower numbers of young Android phone users in the UK. In contrast, the cost of accessing mobile data in South Africa may be prohibitive for many young people. We have made allowances for this by subsidizing cellular data costs; however, this introduces an incentive for participation in the study and introduces potential confounding in the enrolment and engagement analyses.

With respect to the qualitative substudy, we anticipate that conducting deliberative democracy sessions on data governance models with young people who may not necessarily be familiar with the scientific literature may be another challenge. To mitigate this, we have prepared detailed recruitment strategies for each context and co-designed the educational material for deliberative democracy sessions along with youth advisors and YPAGs in each of the sites. Mock qualitative sessions were also held with YPAGs by the youth advisors to ensure the sessions planned were useful, accessible, and enjoyable for participants.

Limitations

The primary potential limitation is that the views of those who use the MindKind app and participate in the deliberative democracy sessions in the UK, India, or South Africa may not be representative of young people as a whole in these countries or elsewhere. For example, due to the restrictions on the quantitative substudy, the study participants will include young adults that have access to the internet on an Android device and are able to interact with the digital platform in English. The sites made accommodations for providing access to data/internet for youth to engage with the study application, for those that needed it. The qualitative substudy allows for flexibility on the language, on access to a device, and on the interaction with the study application, thereby allowing for a more diverse group of youth to opine on the study questions. This potential difference in representativeness is, however, expected and understanding these potential differences is part of this study. Moreover, the triangulation of data sources (app, group sessions, youth advisors’ feedback) will enable the consideration of similarities and differences between various sub-groups of young people. The study was not designed to assess mental health outcomes, and therefore any analyses of the data pertaining to these outcomes must be interpreted with caution.

Additionally, the engagement question posed in this study is limited to the effects of AI choice and governance model on participation. This is not a thorough study of optimal app-based engagement strategies. While we have incorporated youth feedback with respect to governance models, AI topics, survey and study design, website and app designs, and user interfaces, a more iterative user-centered design approach should be incorporated in future iterations of the study. Additionally, we have not explored other engagement strategies such as in-app or text message notifications, gamification or other methods of increasing engagement, though these would be important factors to consider in future efforts.

Data availability

Underlying data

No underlying data are associated with this article.

Extended data

Synapse: MindKind Databank

Appendix 1: (Democratic choice voting options, PDF format)

https://doi.org/10.7303/syn26067677.1

Appendix 2: (Daily AI questions, PDF format)

https://doi.org/10.7303/syn26067678.1

Appendix 3: (Eligibility and Baseline questionnaires, PDF format)

https://doi.org/10.7303/syn26067679.1

Data are available under the terms of the Creative Commons 1.0 “Universal” ( CC0 1.0 public domain dedication)

Many thanks for addressing all the concerns and revising the manuscript.

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Global Mental Health, Implementation Research, Yoga and Mental Health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard.

While I find the overall study very insightful and compelling, the language of the paper was a bit concerning. The feasibility study is embedded in a mixed-methods framing of a pilot 12-week app-based sub-study nested within the willingness to engage as well as understanding the preferences of youth. It is in both these domains that I find the study lacking. It appears very dominated by trial language and the process indicators and process as an ethical ontological and phenomenological problem hasn't been teased out. I find the use of the option 'democracy' limiting and mislabelled. Does the conceptual basis of democracy imply greater access to the use of data or say in the generation of data? Could the frame of reference for acceptability be improved?

Ultimately, how issues of power, access to information provided to secure funding, and advancing advocacy and research, addressing data protection and data security from national/LMIC/youth perspectives, I am not convinced that quantitative substudy needs a study design section for the governance model. More clarity is needed here.

My general feedback is that while the overall questions on choices are highly regimented, I want to know if these are ideas that were generated by the various consultations or will these be tweaked as further advisory boards and consultations are carried out.

Is the study design appropriate for the research question?

Partly

Is the rationale for, and objectives of, the study clearly described?

Partly

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Global mental health, youth mental health, global health partnerships and research

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

Response: Within the MindKind app, there are four data governance arms to which participants are randomized, one of which is labeled within this paper as “democracy” because each participant has equal say in choosing the terms of data sharing. However, within the app itself, we do not use the term “democracy”; youth participants are not exposed to this potentially biasing term. This said, we do use the term “vote” as we do ask participants to vote on their preferred data governance choices. For example, the language for the Option D (Democracy + Server Access) is as follows: “You and other participants get to vote how researchers can access and use your data. Researchers will only view your data on a secure server - they cannot download your data.  Before researchers can use the data, we will tell you the voting results.” ​​​​​​​Response: There have been no studies to date that have directly examined a multinational cohort of young peoples’ preferences with regard to big health data governance. In this study, we approach assessing young peoples’ preferences for data governance both through qualitative (deliberative democracy) and quantitative (via our pilot app) methods. Our vision is that these two data streams together will provide powerful insights: from the deliberative democracy sessions understanding of young peoples’ perception of, feelings and opinions toward, and preferences with regards to data governance, as well as how this grounding translates (or doesn’t) into big health data research participation as captured by the pilot app. We have added Table 2 to better highlight the information gained from each substudy with respect to data governance.    ​​​​​​​Response: Youth leads and youth panel members were involved in the study design and development of data collection tools from early on in protocol development. We elicited their feedback in the survey questions about active ingredients and mental health indicators, the data governance models for the randomized study arms, and the qualitative study design. All procedures described in this protocol manuscript are inclusive of youth lead and youth panel feedback. We will likely not make further modifications to our questionnaires or study design, but will continue to have ongoing meetings with youth leads and youth panel members and will be flexible in incorporating any perspectives on revised procedures as our resources and timelines allow. Table 2 outlines the roles played by the advisory boards in the final designs.

While I find the overall study very insightful and compelling, the language of the paper was a bit concerning. The feasibility study is embedded in a mixed-methods framing of a pilot 12-week app-based sub-study nested within the willingness to engage as well as understanding the preferences of youth. It is in both these domains that I find the study lacking. It appears very dominated by trial language and the process indicators and process as an ethical ontological and phenomenological problem hasn't been teased out. I find the use of the option 'democracy' limiting and mislabelled. Does the conceptual basis of democracy imply greater access to the use of data or say in the generation of data? Could the frame of reference for acceptability be improved?

Ultimately, how issues of power, access to information provided to secure funding, and advancing advocacy and research, addressing data protection and data security from national/LMIC/youth perspectives, I am not convinced that quantitative substudy needs a study design section for the governance model. More clarity is needed here.

My general feedback is that while the overall questions on choices are highly regimented, I want to know if these are ideas that were generated by the various consultations or will these be tweaked as further advisory boards and consultations are carried out.

The protocol paper aims to describe the MindKind study, a technology approach to mental health intervention for young people. This protocol focuses on understanding the data governance and engagement strategies influencing young peoples’ (aged 16-24) participation in app-based mental health promotion/studies. This study applies mixed methods and consists of two substudies.

Overall, this paper has several strengths. It is timely and targets several important research needs in applying technology for young people’s mental health promotion. Conducting the study in three countries and utilizing a global research network (including young people from multiple countries) are innovated. However, the writing is not always clear. The paper can be improved by better organization and better defining research questions. The research questions proposed to be studied are not always clear (e.g., questions are presented differently in different sections), and some information presented may not be in the right section. Below I list suggestions for improving organization/readability/clarity.

INTRODUCTION. METHODS.

The fourth paragraph is confusing. The paragraph starts with describing methods and hypotheses before describing the MindKind Project and its aims. It will be much clearer if the order is reversed. The authors should also clarify whether the protocol paper’s aims are the same as the MindKind Project Aims. The goals/aims presented in several sections don’t seem to be the same.

To me, it seems that MindKind is to study two areas of questions related to technology use in young people’s mental health promotion: (1) One is to study engagement strategies for mental health app development. Through a purpose-built app, the team will study the feasibility and acceptability of the app approach/strategies (using a mixed-method design), as well as understand potential influential factors and mechanisms. (2) The other is to study data collection/governing/sharing models (i.e., to identify acceptable approaches for young people to report their mental health data/contribute to databank using app data collection; to understand young people’s preferred ways for governance/sharing their data with research scientists).

The MindKind mental health app considers 5 active ingredients (sleep, body movement, social connections, positive activities). It will be helpful to introduce the framework/concept in the introduction as part of the MindKind overview first (rather than describe this in the method and mix with other design info), and then introduce this protocol will test an App engagement approach (by studying whether given-option-to-choose would be more acceptable than the fix-sequence approach using an RCT/quantitative design).

MindKind is guided by the Youth Engagement/partnership approach. It might be helpful to briefly describe this approach as part of the MindKind project overview in the Introduction section (e.g., moving the descriptions from method to introduction & using a figure to present the structure).

I would also suggest moving detailed descriptions about the youth group & research structure in the appendix. The current presentation format is a bit distractive because of too much info & not fully integrated with the two substudies.

For clarity, it will be helpful to organize the method section by research questions (engagement strategies for app design vs. data governance) rather than by types of study (qualitative vs quantitative). The current presentation approach is a bit confusing. When reading each substudy, it’s not quite clear what research questions the team is trying to study. For a research paper, it tends to be easier to understand if the goals/objectives are presented first, followed by presenting methods used to answer the questions.

Please clearly state the research questions at the beginning of each substudy section, and describe data collection methods/measures separately for each research question. There are several research questions studied in each substudy, but the method for each research question is not clear. Using a table can help.

For each substudy, please clarify how youth-researchers (or different youth group structures) will participate in each substudy (e.g., do they help design, recruit, analyze data). How are they working with the expert researchers?

What is the age range of the adolescents to be studied?

Figure 2. There are 4 active ingredient areas, but intervention arm 2 only includes 3 ingredients. Why?

The study timeline is not clear. Would quantitative or qualitative studies be conducted simultaneously or sequentially? Would quantitative data be used to inform the qualitative study?

Figure 3 is a very intense data collection procedure. Is this part of the co-design with the youth? Would this be modified if the youths do not think this approach will work?

Is the study design appropriate for the research question?

Partly

Is the rationale for, and objectives of, the study clearly described?

Partly

Are sufficient details of the methods provided to allow replication by others?

Partly

Are the datasets clearly presented in a useable and accessible format?

Partly

Reviewer Expertise:

Child and adolescent mental health research (epidemiology, mHealth intervention development, and testing, intervention implementation/user-center testing)

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

The protocol paper aims to describe the MindKind study, a technology approach to mental health intervention for young people. This protocol focuses on understanding the data governance and engagement strategies influencing young peoples’ (aged 16-24) participation in app-based mental health promotion/studies. This study applies mixed methods and consists of two substudies.

Overall, this paper has several strengths. It is timely and targets several important research needs in applying technology for young people’s mental health promotion. Conducting the study in three countries and utilizing a global research network (including young people from multiple countries) are innovated. However, the writing is not always clear. The paper can be improved by better organization and better defining research questions. The research questions proposed to be studied are not always clear (e.g., questions are presented differently in different sections), and some information presented may not be in the right section. Below I list suggestions for improving organization/readability/clarity.

INTRODUCTION. Response: Thank you for this observation. We have removed the discussion of the overarching program aims and focused more on the specific aims of the (sub)studies described here. We have re-written this paragraph and clarified the study aims there, in the Study Overview section as well as the individual substudy sections per your suggestions that follow. ​​​​​​​Response: See response above. ​​​​​​​Response: While the AIs are an integral part of the participants’ experience in the app, they are a secondary concept relative to the overall aims, and only specifically related to the quantitative substudy. As such, we believe that this information would actually detract from the message of the introduction, rather than add to it. We have improved the description of the four AIs in the Study Overview section.  ​​​​​​​Response: Thank you for this suggestion. We have added these details to the introduction section, as follows:

The fourth paragraph is confusing. The paragraph starts with describing methods and hypotheses before describing the MindKind Project and its aims. It will be much clearer if the order is reversed. The authors should also clarify whether the protocol paper’s aims are the same as the MindKind Project Aims. The goals/aims presented in several sections don’t seem to be the same.

To me, it seems that MindKind is to study two areas of questions related to technology use in young people’s mental health promotion: (1) One is to study engagement strategies for mental health app development. Through a purpose-built app, the team will study the feasibility and acceptability of the app approach/strategies (using a mixed-method design), as well as understand potential influential factors and mechanisms. (2) The other is to study data collection/governing/sharing models (i.e., to identify acceptable approaches for young people to report their mental health data/contribute to databank using app data collection; to understand young people’s preferred ways for governance/sharing their data with research scientists).

The MindKind mental health app considers 5 active ingredients (sleep, body movement, social connections, positive activities). It will be helpful to introduce the framework/concept in the introduction as part of the MindKind overview first (rather than describe this in the method and mix with other design info), and then introduce this protocol will test an App engagement approach (by studying whether given-option-to-choose would be more acceptable than the fix-sequence approach using an RCT/quantitative design).

MindKind is guided by the Youth Engagement/partnership approach. It might be helpful to briefly describe this approach as part of the MindKind project overview in the Introduction section (e.g., moving the descriptions from method to introduction & using a figure to present the structure).

“Throughout the project we centre our work on the voices of young people, by grounding our work in a youth-adult participatory research approach whereby we seek to involve youth stakeholders and researchers as equal partners. There are several ways for young people with lived experiences of mental health issues to be involved in our MindKind work, including as youth leads who are employed as members of the study staff, as in-country youth panel members, as multi-national youth panel members, and as research participants.” ​​​​​​​Response: While we agree that this information is not common in protocol papers such as these, we feel that this is an important and somewhat novel feature of our project and study methodology that would be minimized by relegating it to an appendix stored in an external data repository. We have substantially shortened the descriptions in the text and moved this information to Table 1. We believe that this compromise improves the readability of the paper while maintaining the focus on youth involvement.

I would also suggest moving detailed descriptions about the youth group & research structure in the appendix. The current presentation format is a bit distractive because of too much info & not fully integrated with the two substudies.

METHODS. ​​​​​​​Response: We respectfully disagree with your suggestion that the paper clarity would be improved by organizing the paper by question versus substudy. We have added language at the beginning of each substudy section, reintroducing the substudy aims as well as added a table (Table 2) describing the research questions, and how they’re addressed by each of the substudies. We hope this will serve as a guidepost to help readers better understand the relationship between the substudies. ​​​​​​​Response: Thank you for this suggestion. As noted above, we have added a table describing the study questions and how they are addressed in each substudy, as well as reintroducing the substudy aims in each section. ​​​​​​​Response: We have included this information in Table 2. ​​​​​​​Response: This is outlined in the first paragraph of the MindKind Study Design section and pertains to both the Quantitative and Qualitative substudies:

For clarity, it will be helpful to organize the method section by research questions (engagement strategies for app design vs. data governance) rather than by types of study (qualitative vs quantitative). The current presentation approach is a bit confusing. When reading each substudy, it’s not quite clear what research questions the team is trying to study. For a research paper, it tends to be easier to understand if the goals/objectives are presented first, followed by presenting methods used to answer the questions.

Please clearly state the research questions at the beginning of each substudy section, and describe data collection methods/measures separately for each research question. There are several research questions studied in each substudy, but the method for each research question is not clear. Using a table can help.

For each substudy, please clarify how youth-researchers (or different youth group structures) will participate in each substudy (e.g., do they help design, recruit, analyze data). How are they working with the expert researchers?

What is the age range of the adolescents to be studied?

“To participate in the MindKind study, youth participants must live in one of the participating countries and be legally able to provide consent (age 16–24 in the UK or 18–24 for India and South Africa).“ ​​​​​​​Response: The study consists of 3 4-week rotations on a single topic (AI), so participants will not participate in all AIs regardless of which arm they are in. We have amended the figure captions to try to provide more clarity:

Figure 2. There are 4 active ingredient areas, but intervention arm 2 only includes 3 ingredients. Why?

“In this example, the ARM 1 participant has selected the topics Body movement, Positive activities and Social connections for their topics, while the ARM 2 participant has been randomly assigned to Sleep, Positive activities and Body movement for their 3 topics.” ​​​​​​​Response: Both substudies are being conducted simultaneously as two-pronged approaches to answering the questions about youths’ feelings, preferences, and concerns about data governance structures for sharing their mental health data, as well as their willingness to participate. We will synthesize the results from both substudies to form our conclusions. It is also important to note that the choice of governance models used in the quantitative study have been selected based on feedback from the youth advisory panels, so have already been informed by the voices of the youth. We have added the following text for clarity:

The study timeline is not clear. Would quantitative or qualitative studies be conducted simultaneously or sequentially? Would quantitative data be used to inform the qualitative study?

“The two substudies will run concurrently, and results from the two will be compared to understand participants’ ideals and their effect on participation relative to current research standards.” ​​​​​​​Response: The data collection process described in Figure 3 was developed in collaboration with the professional youth advisors and youth advisory panels. We presented three different scenarios to the youth groups and incorporated their feedback into the design of the survey. This survey cadence presented in Figure 3 was deemed the least burdensome. Youth groups also informed strategies to maximize engagement with surveys, such as reflection exercises and GIFs for performing surveys. These strategies have also been used successfully in other studies (see Nickels et al cited below*).

Figure 3 is a very intense data collection procedure. Is this part of the co-design with the youth? Would this be modified if the youths do not think this approach will work?

The survey approach consisted of a brief daily survey that asked one question about mood and 1 to 5 questions about the active ingredient to be studied. It also included one weekly survey of mood standard instruments as well as the selected or assigned active ingredient. We timed the length of each survey type and found that on average daily surveys took no more than 5 minutes to complete in total, with a range of 30 secs to 5 minutes, with the weekly survey taking an average of 9 minutes, with a range of 3-22 minutes.

The last week of the study consists of a remote user survey to determine what aspects of the experiment were burdensome, which engagement strategies were deemed helpful in motivating youth to complete the survey, and ideas from youth to improve the survey experience. A similar survey is also administered when a participant asks to leave the study or has not completed a survey after 1 week. This information will be used to improve the survey experience.

We have added a brief statement describing these efforts in the manuscript.

* Nickels S, Edwards MD, Poole SF, Winter D, Gronsbell J, Rozenkrants B, Miller DP, Fleck M, McLean A, Peterson B, Chen Y, Hwang A, Rust-Smith D, Brant A, Campbell A, Chen C, Walter C, Arean PA, Hsin H, Myers LJ, Marks WJ Jr, Mega JL, Schlosser DA, Conrad AJ, Califf RM, Fromer M. Toward a Mobile Platform for Real-world Digital Measurement of Depression: User-Centered Design, Data Quality, and Behavioral and Clinical Modeling. JMIR Ment Health. 2021 Aug 10;8(8):e27589. doi: 10.2196/27589. PMID: 34383685; PMCID: PMC8386379.

This study protocol by the MindKind consortium describes a mixed-methods approach to explore the data governance models that will influence the engagement of young people in mobile app-based mental health studies. This is a fairly well-written protocol. There are a few concerns which the authors may wish to address in the next version.

Participants will be randomized to one of the two engagement strategies as illustrated in Figure 2. Are there any other strategies as well to engage participants and improve compliance? This is important to know as the engagement with the mobile app will be determined by a multitude of factors. One of these is the way in which the modules are presented, and the other is the data governance structure. What other factors which affect engagement and compliance are measured in this study? One example is user interface and user experience. There could be multiple other factors that can be important confounders. Please clarify.

On page 10, paragraph on governance acceptability, it is mentioned that the effect of the governance model on study enrolment will be assessed. It will also be good to study the effect of the governance model on engagement and compliance. This is probably mentioned in the next paragraph. Please confirm.

Will the participants randomized to Group B-D be aware of different types of data governance structures? Or will they be randomly allocated to the group first and then they will be only informed about the allocation? This will have important implications in terms of their enrolment and engagement.

Baseline awareness about the rights of an individual on their personal data will play an important role in determining the extent to which participants contribute to deliberate democracy approach. Hopefully, the educational materials will cover this aspect and provide information to the participants.

How challenging is it to organize deliberative democracy sessions as an online group?

Is there a plan to triangulate the data from the two approaches? If yes, it will be good to add a short description about the same.

Please describe how the data gathered in the study will be protected.

Professional Youth Advisors will be full-time hired by the project. Similarly, the members of the Youth Panel Advisory Groups and International Youth Panel members will also be hired full-time? If not, will they receive any honorarium for their participation? Participants in the qualitative sub-study will also spend a good amount of their time - will they be remunerated for the same?

Is the study design appropriate for the research question?

Yes

Is the rationale for, and objectives of, the study clearly described?

Yes

Are sufficient details of the methods provided to allow replication by others?

Yes

Are the datasets clearly presented in a useable and accessible format?

Not applicable

Reviewer Expertise:

Global Mental Health, Implementation Research, Yoga and Mental Health

I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.

This study protocol by the MindKind consortium describes a mixed-methods approach to explore the data governance models that will influence the engagement of young people in mobile app-based mental health studies. This is a fairly well-written protocol. There are a few concerns which the authors may wish to address in the next version. Response: For the purposes of this study, we are only perturbing data governance and selection of AIs vs fixed experience. You are correct that there are a number of factors that may improve governance, but these are not explored exhaustively here. We would love to do that in the future, but it was not feasible to do so in this pilot. We have added a paragraph to the Limitations section of the Discussion to this effect:

1. Participants will be randomized to one of the two engagement strategies as illustrated in Figure 2. Are there any other strategies as well to engage participants and improve compliance? This is important to know as the engagement with the mobile app will be determined by a multitude of factors. One of these is the way in which the modules are presented, and the other is the data governance structure. What other factors which affect engagement and compliance are measured in this study? One example is user interface and user experience. There could be multiple other factors that can be important confounders. Please clarify.

“Additionally, the engagement question posed in this study is limited to the effects of AI choice and governance model on participation. This is not a thorough study of optimal app-based engagement strategies. While we have incorporated youth feedback with respect to governance models, AI topics, survey and study design, website and app designs, and user interfaces, a more iterative user-centered design approach should be incorporated in future iterations of the study. Additionally, we have not explored other engagement strategies such as in-app or text message notifications, gamification or other methods of increasing engagement, though these would be important factors to consider in future efforts.” Response: Thank you for pointing this out. While this is mentioned in the Data Analysis Plan for Engagement, this is easily overlooked. We have added a sentence at the end of the Study Design: governance section to highlight this intention:

2. On page 10, paragraph on governance acceptability, it is mentioned that the effect of the governance model on study enrolment will be assessed. It will also be good to study the effect of the governance model on engagement and compliance. This is probably mentioned in the next paragraph. Please confirm.

“We will also assess the effect of governance models on study engagement and retention (See the Data Analysis Plan Engagement section for details).” ​​​​​​​Response: Participants in B-D will have no knowledge of other potential governance structures. They will be presented with an informed consent outlining the data governance for the model to which they were randomized, without mention of other alternatives. In this way, we mirror the experience they would have in a typical research study. We have clarified this in the text:

3. Will the participants randomized to Group B-D be aware of different types of data governance structures? Or will they be randomly allocated to the group first and then they will be only informed about the allocation? This will have important implications in terms of their enrolment and engagement.

“In order to mirror the experience in a typical study, participants will be exposed to an informed consent specific to their data governance model and can choose to either join or not. They will have no exposure to other potential governance models.” ​​​​​​​Response: Prior to their first deliberative democracy session, participants are instructed to watch two video modules that contain comprehensive, engaging information about different models of data governance used to organize research data, including addressing the rights of individuals. A formal manuscript describing the educational materials in detail will be forthcoming. ​​​​​​​Response: Organizing deliberative democracy sessions online has posed a number of challenges and opportunities. Briefly, pragmatic challenges include multinational scheduling, the length of online sessions (reasonable attention spans), and technology access issues. Conceptual challenges include educational provision and the roles of facilitators. Given that this is a protocol paper, we don’t feel it appropriate to add these retrospective learnings here, however, we will discuss these challenges and opportunities further in our forthcoming analysis paper. ​​​​​​​Response: Yes, we have added the following sentence to the “Interplay between quantitative and qualitative studies” section:

4. Baseline awareness about the rights of an individual on their personal data will play an important role in determining the extent to which participants contribute to deliberate democracy approach. Hopefully, the educational materials will cover this aspect and provide information to the participants.

5. How challenging is it to organize deliberative democracy sessions as an online group?

6. Is there a plan to triangulate the data from the two approaches? If yes, it will be good to add a short description about the same.

“Results from the two substudies will be compared to understand participants’ ideals and their effect on participation relative to current research standards.” ​​​​​​​Response: We added a Data Protection section in both the Quantitative and Qualitative Study sections to describe the security and privacy measures to safeguard the confidentiality and integrity of the data and prevent improper access. ​​​​​​​Response: Youth panel members participate in two meetings monthly + occasional asynchronous assignments. This information has been added to the newly added Table 1. Members are either paid an honorarium or hourly, depending on the site.

7. Please describe how the data gathered in the study will be protected.

8. Professional Youth Advisors will be full-time hired by the project. Similarly, the members of the Youth Panel Advisory Groups and International Youth Panel members will also be hired full-time? If not, will they receive any honorarium for their participation? Participants in the qualitative sub-study will also spend a good amount of their time - will they be remunerated for the same?

Qualitative substudy participants receive incentives for participation in deliberative democracy sessions. In the UK, participants were offered a 30GBB gift card for each session in which they participated (maximum compensation 60GBP). In India, participants were offered a single 500INR gift card for all deliberative session participation (maximum compensation 500INR). In South Africa, deliberative session participants received a 400ZAR airtime voucher per session (maximum compensation 800ZAR). Additionally, South African participants of the quantitative study received a 150ZAR airtime voucher per month of participation. This information has been added as follows:

“Given the amount of time required for pre-learning activities and participation, qualitative substudy participants will receive a modest incentive for participation in the form of a gift card (UK, India) or airtime voucher (South Africa).”