Effectiveness of recruitment to a smartphone-delivered nutrition intervention in New Zealand: analysis of a randomised controlled trial.

OBJECTIVES: Delivery of interventions via smartphone is a relatively new initiative in public health, and limited evidence exists regarding optimal strategies for recruitment. We describe the effectiveness of approaches used to recruit participants to a smartphone-enabled nutrition intervention trial.
METHODS: Internet and social media advertising, mainstream media advertising and research team networks were used to recruit New Zealand adults to a fully automated smartphone-delivered nutrition labelling trial (no face-to-face visits were required). Recruitment of Māori and Pacific participants was a key focus and ethically relevant recruitment materials and approaches were used where possible. The effectiveness of recruitment strategies was evaluated using Google Analytics, monitoring of study website registrations and randomisations, and self-reported participant data. The cost of the various strategies and associations with participant demographics were assessed.
RESULTS: Over a period of 13 months, there were 2448 registrations on the study website, and 1357 eligible individuals were randomised into the study (55%). Facebook campaigns were the most successful recruitment strategy overall (43% of all randomised participants) and for all ethnic groups (Māori 44%, Pacific 44% and other 43%). Significant associations were observed between recruitment strategy and age (p<0.001), household size (p<0.001), ethnicity (p<0.001), gender (p=0.005) and interest in healthy eating (p=0.022). Facebook campaigns resulted in the highest absolute numbers of study registrations and randomisations (966 and 584, respectively). Network strategies and Facebook campaigns cost least per randomised participant (NZ$4 and NZ$5, respectively), whereas radio advertising costs most (NZ$179 per participant).
CONCLUSION: Internet and social media advertising were the most effective and least costly approaches to recruiting participants to a smartphone-delivered trial. These approaches also reached diverse ethnic groups. However, more culturally appropriate recruitment strategies are likely to be necessary in studies where large numbers of participants from specific ethnic groups are sought. TRIAL REGISTRATION: ACTRN12614000644662; Post-results. © Article author(s) (or their employer(s) unless otherwise stated in the text of the article) 2017. All rights reserved. No commercial use is permitted unless otherwise expressly granted.

RCT Entities:   Population Interventions Outcomes

Reports on effectiveness and costs of a range of recruitment strategies (internet and social media advertising, mainstream media advertising and research team networks) were used for a smartphone-delivered study.

Effectiveness was evaluated using Google Analytics, monitoring of study website registrations and randomisations and self-reported data.

Information is provided on participant retention rates and strategies to reduce duplicate and fake registrations.

The broad categorisation of self-reported recruitment source data prevented more precise subcategory analysis.

Staffing costs associated with recruitment strategies were not assessed.

Introduction

Monitoring and reporting of study recruitment strategies and their effectiveness facilitates improvements in design and methods for future randomised controlled trials (RCTs). Systematic reviews provide an indication of successful recruitment strategies and trial design elements such as incentives and open-label design,1 2 and their cost-effectiveness.3 However, applicability and success of various strategies depends on specifics of the trial design, setting and study population.

Smartphone delivery and measurement of trial interventions4–6 is a relatively new initiative and presents unique recruitment challenges compared with traditional RCTs. In addition to standard trial eligibility criteria, prospective participants must have smartphone access and be adept at using technology. Moreover, attrition rates are potentially higher because enrolment rates may be augmented by a ‘novelty factor’ that decays over time.7 Lack of personal contact with study staff may also increase attrition.7 Such challenges are also relevant to public health and health promotion interventions that increasingly deliver programme components using smartphone technology. Identifying successful recruitment and retention strategies for this type of intervention is, therefore, timely and important.

In addition to traditional RCT recruitment methods such as advertising via community fliers, newspapers or media,8 newer strategies including Internet and social media advertising9 are being increasingly used. A systematic review found that online recruitment strategies, Facebook in particular, were promising ways to recruit participants for internet and mobile health (mHealth) studies.7 However, limited data were available, and specific gaps identified were reporting of participant retention rates, and methods to identify and manage fake and duplicate registrations.

Potential greater reach among underserved and diverse population groups is one suggested benefit of online recruitment.8 10 In New Zealand (NZ), Māori (indigenous New Zealanders making up 14.9% of population) and Pacific peoples (7.4%) are priority groups for health intervention programmes due to their disproportionately high prevalence of non-communicable diseases and associated risk factors.11 Adequate representation of these groups in health research is vital but often hard to achieve with generic recruitment strategies, so ethnic-specific tailoring of strategies is recommended.12 13

Our aim was to assess the effectiveness of a range of approaches used to recruit participants to a large smartphone-delivered nutrition labelling RCT14 and to examine associations of recruitment strategy with ethnicity and other participant demographics.

Methods

The Starlight RCT received ethics approval from the University of Auckland Human Participants Ethics Committee (reference number 011390), and the study protocol was published in 2014.15 The trial was registered on the Australian New Zealand Clinical Trials Registry (ACTRN12614000644662). All participants provided informed consent via a questionnaire completed on their smartphone.

The aim of the trial was to evaluate the effects of two interpretive nutrition labels, traffic light labels and Health Star Rating labels, compared with a non-interpretive label on the healthiness of consumer food purchases. The target was to recruit and randomise 1500 eligible NZ adults, composed of approximately equal numbers of Māori, Pacific and other ethnicities. Eligible participants were aged 18 years or older, owned a smartphone, were the main household shopper and could read and understand English. All components of the trial (screening, consent, registration, randomisation, intervention and data collection) were delivered via a bespoke, automated smartphone application (app).16 Outcome data were collected in the form of scanned household food purchase records. Completion of a run-in period (week 1) with a requirement to record at least 15 food items was a prerequisite for randomisation. The 4-week nutrition labelling intervention comprised randomisation to one of two interpretive nutrition labels (traffic lights17 or Health Star Ratings18) or to a control group (nutrition information panel).19 The primary study outcome was healthiness of all foods and beverages purchased over the intervention period, measured using the Food Standards Australia New Zealand nutrient profiling scoring criterion.20

Recruitment

Recruitment commenced in October 2014. A key focus was to attract 500 Māori and 500 Pacific participants, with advice provided by Māori and Pacific team members. Recruitment was open initially to all ethnic groups, but was closed to non-Māori and non-Pacific individuals after 6 months when the target for this group (500 participants) was reached. Recruitment of Māori and Pacific individuals continued for a further 5 months, but at that time it became clear that recruitment targets would not be met within the study timeframes. Therefore, recruitment was reopened to all ethnicities again for 2 months and closed finally in November 2015.

Recruitment campaigns directed volunteers to a designated trial website to answer a prescreening questionnaire, and a link to download the trial app was then provided. Enrolment occurred via the app, but the prescreening step allowed the study team to monitor registration numbers and ethnicity targets. The trial app was available free of charge in NZ Google Play and iTunes app stores. Information in the app stores also encouraged volunteers to visit the trial website before downloading the app.

Recruitment materials and advertisements were developed by the research team (see online supplementary file1 A, B). Advertising campaigns were conducted at staggered intervals over the 13-month recruitment period using research team networks, Internet and social media advertising and mainstream media advertising (figure 1). The impact of recruitment strategies was assessed continuously by comparison with monthly recruitment targets.

Figure 1

Recruitment rates in response to implementation of key recruitment strategies over time.

Three media releases were issued by the university communications office (October 2014, November 2014 and 2015). Promotion via research team networks was undertaken using group email lists, word of mouth, staff intranets, personal Facebook pages and hard copy recruitment flier distribution. Electronic recruitment fliers were circulated via email lists at several NZ universities, healthcare providers and relevant non-government organisations. Requests were made to NZ schools, particularly those with high number of Māori and Pacific students, to promote the study via their newsletters with the aim of recruiting parents of students. Paid internet advertising was undertaken using 14 promoted posts on Facebook (see online supplementary file 2), two Google AdWords campaigns (see online supplementary file 3 part 1 and part 2) and one LinkedIn campaign. Other paid advertising included an advertisement in a Māori magazine (including a web banner on their website), promotional advertisements on Pacific radio and representation at relevant events, such as Māori and Pacific cultural festivals and health provider conferences.

Strategies to maintain retention

Participants were sent regular notifications via the study app, SMS text reminders16 and email messages. All registered participants received a NZ$10 gift voucher, and those who completed the trial (defined as completion of the follow-up questionnaire at the end of the 5-week study) received a further NZ$80 gift voucher.

Registration logic checks

Because incentives (vouchers) were offered for study participation, repeat registrations by the same participants and re-registration by ineligible participants were identified as risks. To prevent duplicate registrations, individual internet protocol address and email address checks were implemented on the study screening website. Within the app, logic checks for unique and valid email address and valid age (18–100 years) were also implemented. All registrations were checked daily for duplicate surnames, street addresses and phone numbers. All suspected duplicate registrations were investigated by research staff. Confirmed duplicate registrations were contacted with the request that they discontinue the trial.

Data analysis

Information on the demographic characteristics of trial participants was collected via a baseline questionnaire completed in the study app.15 Self-reported ethnicity was grouped into three categories: Māori, Pacific and Other. Google Analytics21 was used to track visits to the study website. Data on self-reported recruitment strategy that attracted participants were collected using the question “How did you find out about the study?” The effectiveness of recruitment strategies was assessed by collating number of participants registered, randomised and completing the trial. Conversion to randomisation was defined as proportion of registered participants randomised, and retention rate was defined as proportion of randomised participants retained. Participants who could not be randomised due to technical issues with the app or their phone were excluded from analysis.

Self-reported recruitment data were matched with recorded costs for each strategy. Only broad categories were available in the self-reported data. Thus in the cost analysis, promotion or advertising via community events, research team’s personal and professional network, and coverage in mainstream paper media were combined in one category titled ‘network and paper media’ (table 1). It was not possible to directly account for the cost of research staff time on each of the strategy, and thus only the direct costs of each strategy were assessed. Costs were reported as total per strategy, and cost-per-randomised and cost-per-completed participant (NZ$).

Table 1

Study recruitment strategies

Recruitment campaign	Subcategories	Corresponding response in baseline survey
Print media and team networks	Personal and professional networks, schools, universities, healthcare providers, non-government organisations, print media, community events and advertising in magazines	Word of mouth Email invitations Supermarket advertising Newspaper or newsletter Other
Internet and social media	Facebook, LinkedIn and Google Ads	Internet
Radio	Advertising on Pacific Radio	Radio

The analysis was conducted in IBM SPSS Statistics V.21.0. Simple descriptive statistics were used to describe the number of participants registered and randomised by recruitment strategy, and key demographics. Statistical difference between categories was tested using the χ2 test for categorical variables and analysis of variance for continuous variables. Statistical tests were two sided at 5% significance level.

Results

Recruitment summary

There were 2448 study registrations, of which 1035 were excluded prior to randomisation. Reasons for exclusion were ineligibility based on study inclusion criteria (n=205), failure to complete the study run-in phase (n=727), duplicate registration due to a technical problem with the app (n=47) and non-randomisation of eligible individuals due to a technical problem with the app (n=56). A further 56 individuals were excluded from analysis because they were randomised in error (ie, randomised even though they failed to meet qualifying run-in criteria) as a result of a technical problem with the app. Thus, 1357 individuals (55% registrations) were randomised and included in the main study data analysis: 243 (18%) Māori, 87 (6%) Pacific and 1027 (76%) other ethnic groups. Of those randomised, 1202 (89%) completed the study: 201 Māori (83%), 75 Pacific (86%) and 926 (90%) other.

The demographic characteristics of the randomised participants, overall and by recruitment strategy, are presented in table 2. The study population was predominantly female (89%) with a mean (SD) age of 33 (9) years. Representation of Māori, Pacific and other ethnicities was similar to their distribution in the NZ population (table 2). However, more than two-thirds of trial participants were tertiary educated (vs 26% in 2013 NZ Census),22 and nearly all (97%) reported that they were moderately to very interested in healthy eating. Household income was also not a representative of the general population, with higher income groups over-represented in the sample (table 2).

Table 2

Demographic characteristics of study participants according to the recruitment strategy

	All study participants (n=1357)	By recruitment strategy*						NZ population according to 2013 Census
		Internet (n=584)	Radio (n=14)	Word of mouth (n=343)	Email invitation (n=150)	Newspaper/newsletter (n=111)	Other (n=151)
Age; mean (SD)	32.7 (9.2)	31.2 (8.6)	36.5 (8.5)	32.3 (9.5)	34.3 (9.7)	37.43 (9.72)	33.83 (8.0)	38.0
Household size, mean (SD)	3.6 (1.7)	3.4 (1.6)	5.3 (2.3)	3.6 (1.7)	3.6 (1.5)	4.1 (1.5)	3.8 (2.1)	2.7
Gender, n (%)
Male	152 (11.2)	54 (9.2)	1 (7.1)	48 (14.0)	26 (17.3)	11 (9.9)	10 (6.6)	48.7%
Female	1205 (88.8)	530 (90.8)	13 (92.9)	295 (86.0)	124 (82.7)	100 (90.1)	141 (93.4)	51.3%
Ethnicity, n (%)
Māori	243 (17.9)	106 (18.2)	4 (28.6)	45 (13.1)	25 (16.7)	23 (20.7)	39 (25.8)	14.9%
Pacific	87 (6.4)	38 (6.5)	10 (71.4)	20 (5.8)	12 (8.0)	3 (2.7)	4 (2.6)	7.4%
Other	1027 (75.7)	440 (75.3)	0	278 (81.0)	113 (75.3)	85 (76.6)	108 (71.5)	74.0%
Income, n (%)†
NZ$20 000 or less	111 (10.4)	61 (13.1)	0	31 (11.4)	8 (6.7)	1 (1.3)	9 (7.1)	11.1%
NZ$20 001 –$40 000	154 (14.4)	59 (12.7)	1 (14.3)	42 (15.4)	22 (18.5)	8 (10.4)	22 (17.5)	20.7%
NZ$40 001 –$60 000	193 (18.1)	81 (17.5)	1 (14.3)	48 (17.6)	25 (21.0)	14 (18.2)	23 (18.3)	15.5%
NZ$60 001 –$70 000	114 (10.7)	56 (12.1)	0	28 (10.3)	10 (8.4)	5 (6.5)	14 (11.1)	7.0%
NZ$70 001 –$1 00 000	284 (26.6)	110 (23.7)	4 (57.1)	71 (26.1)	30 (25.2)	32 (41.6)	37 (29.4)	18.0%
NZ$100 001 or more	213 (19.9)	97 (20.9)	1 (14.3)	52 (19.1)	24 (20.2)	17 (22.1)	21 (16.7)	27.6%
Qualification, n (%)
None	28 (2.1)	10 (1.7)	2 (14.3)	8 (2.3)	3 (2.0)	2 (1.8)	3 (2.0)	18.6%
Secondary school qualification	338 (24.9)	146 (25.0)	6 (42.9)	94 (27.4)	25 (16.7)	32 (28.8)	34 (22.5)	35.6%
University or polytechnic degree/diploma	893 (65.8)	395 (67.6)	3 (21.4)	213 (62.1)	108 (72.0)	67 (60.4)	104 (68.9)	26.0%
Trade certificate	48 (3.5)	16 (2.7)	3 (21.4)	15 (4.4)	6 (4.0)	5 (4.5)	3 (2.0)	8.6%
Other	50 (3.7)	17 (2.9)	0	13 (3.8)	8 (5.3)	5 (4.5)	7 (4.6)	11.1%
Interest in healthy eating, n (%)
Not particularly interested	37 (2.7)	15 (2.6)	1 (7.1)	13 (3.8)	1 (0.7)	1 (0.9)	5 (3.3)	n/a
Moderately to very interested	1320 (97.3)	569 (97.4)	13 (92.9)	330 (96.2)	149 (99.3)	110 (99.1)	146 (96.7)	n/a

*Recruitment source data were missing for n=3 randomised participants.

†Household income data were missing for n=288 randomised participants.

n/a, not applicable; NZ, New Zealand.

Recruitment strategy was significantly associated with participants’ demographic characteristics, namely age (p<0.001), household size (p<0.001), ethnicity (p<0.001), gender (p=0.005) and interest in healthy eating (p=0.022). Participants recruited via internet/social media were younger on average (31 years), whereas those recruited via radio and newspaper advertising were older (36.5 and 37 years, respectively). A larger proportion of male participants was recruited via strategies focused on personal contact, namely word of mouth and email invitations (14% and 17%, respectively, vs 7%–9% by other strategies). Finally, radio advertising attracted a greater proportion of participants not interested in healthy eating (7% vs 1%–4% by other strategies).

Recruitment effectiveness

A cumulative summary of recruitment over time in response to the various strategies is presented in figure 1. The largest peaks in registrations were observed in response to Facebook campaigns (up to 600 new registrations per campaign), followed by promotion via schools and research team networks (50–100 new registrations per campaign).

Analysis of trial registration website visits using data from Google Analytics and self-reported recruitment source is presented in figure 2. ‘Social media’ and ‘paid search’ (Google AdWords) brought 55% of the website visits (figure 2A). This aligns with self-reported data, which showed that almost 50% of registrations were due to ‘internet’ campaigns (figure 2B). Further examination showed 98% of such visits to the study registration page came from Facebook, and the remainder were from other social media and paid search, including Google AdWords. Therefore, all registrations that reported ‘internet’ as a source were considered as arising from Facebook. The second highest website traffic acquisition was via the ‘direct’ channel (figure 2A), that is, direct visits to the trial webiste not redirected from other websites (likely people using the website address provided in fliers, emails, magazines and newspapers). Self-reported data also showed that over 40% registered participants were reached via networks (‘word of mouth’ and ‘email invitations’) and media coverage (‘newspapers and magazines’).

Figure 2

Trial website visit and registration numbers by recruitment strategy.

The effectiveness and cost per strategy are presented in table 3. Facebook (paid campaigns and free posts) resulted in the highest absolute registration, randomisation and study completion numbers, both overall and for each ethnic group. However, higher conversion from registration to randomisation rates were achieved by network/media strategies, such as ‘email invitations’ and ‘word of mouth’ (66%–71% vs 60% achieved by Facebook). There was a significant association between recruitment strategy and conversion to randomisation (p=0.011), but not retention. There was also a significant association between ethnicity and recruitment strategies used (p<0.001).

Table 3

Effectiveness and cost of study recruitment campaigns

Recruitment campaign (data missing for n=4 registered participants)		Participant status	All participants	By ethnicity*			Campaign cost (NZ$)
				Māori	Pacific	Other	total	Per randomised participant	Per completed participant
Internet and social media		Registered, n	966	215	73	668	3047	5	6
		Randomised, n (% of registered)	584 (60)	106 (49)	38 (52)	440 (66)
		Completed, n (% of randomised)	507 (87)	91 (86)	32 (84)	384 (87)
Radio		Registered, n	25	10	15	0	2500	179	192
		Randomised, n (% of registered)	14 (56)	4 (40)	10 (67)	0
		Completed, n (% of randomised)	13 (93)	3 (75)	10 (100)	0
Print media and team networks	Supermarket ad	Registered, n	3	0	0	3	2830	4	4
		Randomised, n (% of registered)	1 (33)	0	0	1 (33)
		Completed, n (% of randomised)	1 (100)	0	0	1 (100)
	Word of mouth	Registered, n	506	76	27	394
		Randomised, n (% of registered)	343 (68)	45 (59)	20 (74)	278 (71)
		Completed, n (% of randomised)	306 (89)	34 (76)	18 (90)	254 (91)
	Email invitation	Registered, n	216	42	19	154
		Randomised, n (% of registered)	150 (69)	25 (60)	12 (63)	113 (73)
		Completed, n (% of randomised)	141 (94)	22 (88)	10 (83)	109 (96)
	Newspaper or newsletter	Registered, n	156	29	8	117
		Randomised, n (% of registered)	111 (71)	23 (79)	3 (38)	85 (73)
		Completed, n (% of randomised)	99 (89)	20 (87)	2 (67)	77 (91)
	Other	Registered, n	228	54	11	161
		Randomised, n (% of registered)	151 (66)	39 (72)	4 (36)	108 (67)
		Completed, n (% of randomised)	132 (87)	30 (77)	3 (75)	99 (92)
Total recruited		Registered, n†	2104	427	152	1501	8377	6	7
		Randomised, n (% of registered)	1357 (64)	243 (57)	87 (57)	1027 (68)
		Completed, n (% of randomised)	1202 (89)	201 (83(	75 (86)	926 (90)

*Ethnicity data were missing for n=24 registered participants.

†Total number of registrations in the table/analysis (n=2104) is less than that recorded in the study (n=2448) because a number of people did not complete the registration questionnaire so data were not available on recruitment strategy for those participants.

NZ, New Zealand.

Promotions via networks and paper media were the least expensive strategy (NZ$4 per randomised participant, table 3), closely followed by Facebook posts (NZ$5 per participant). Radio advertising was the most costly strategy used (NZ$179 per randomised participant).

Discussion

This study describes the effectiveness of a range of recruitment strategies used in a smartphone-delivered nutrition labelling intervention trial. Over a period of 13 months, 2104 participants were registered and provided information on recruitment source, of whom 55% were randomised into the study (n=1357). Facebook campaigns were the most successful recruitment strategy, both overall (43% of all randomised participants) and for Māori and Pacific participants (44% each). Although the conversion rate from registration to randomisation for participants recruited via Facebook was not as high as that achieved by network promotions, the vast reach of Facebook (see online supplementary file 3) and ability to target campaigns by demographics, geographic region and interests led to the greatest absolute number of study registrations (n=966) and randomisations (n=584).

Most types of campaign were used several times during the recruitment period. The Pacific radio campaign and advertising in a Māori magazine (and via their website) were only used once; however (due to cost), during the period that study recruitment was open only to Māori and Pacific. Therefore, although it is possible that other ethnic groups might also have been attracted by these campaigns they would have been deemed ineligible on registration, and so these strategies recruited Māori or Pacific participants exclusively.

The study sample was not a representative of the general population and contained a high proportion of tertiary educated adults (66%). One possible explanation for this is utilisation of the professional networks of the study team and university mailing lists. In addition, most of the paid Facebook campaigns were placed on the University of Auckland Facebook page and while the audience was not limited to the subscribers of this page, interest in University Facebook posts is likely to be higher among tertiary educated adults.

The overall dropout rate was 11% (completion rate 89%), which is lower than the typical >20% dropout rate expected for lifestyle interventions.7 This could have been due to some of the unique characteristics of this study, including the offer of a quite substantial (NZ$80) financial incentive on completion, the relatively short 5-week study duration, the automated nature of the intervention which removed the need to travel to attend appointments and use of a prerandomisation run-in period. A run-in period allowed participants to become familiar with the smartphone technology, potentially selected more dedicated users and allowed exclusion of users with incompatible devices. In total, more than 700 people were excluded following the run-in phase.

Overall the results of this study are consistent with previous findings showing that Facebook is an effective research recruitment method,7 radio advertising is less cost effective23 and combined approaches are best.23 Higher female participation rates, particularly of those with higher income and education levels, are also typical for nutrition research.24 A previous study that compared demographic characteristics of participants recruited via either social media or traditional methods found no difference between groups other than in age, which was younger in the social media group.25 A similar association between age and recruitment strategy was apparent in our study (p<0.001), with a lower mean age among participants recruited via Facebook (31 years), compared with those recruited through newspapers and radio (37 and 36 years, respectively). Our analysis also demonstrated a significant association between recruitment strategy and other important demographic characteristics, such as gender, ethnicity and household size. It is important, therefore, to tailor study recruitment approaches to the target population. Our findings also have broader relevance for the health promotion and public health fields which increasingly uses smartphone technology and applications to deliver and monitor healthcare interventions, for education, and to support consumer behaviour change.26 27

Despite using a wide range of culturally-targeted media-based strategies and additional resources to recruit Māori and Pacific participants, targets were still not met. Face-to-face recruitment building on community networks and connections is a strategy commonly used to recruit Māori and Pacific participants into studies,13 and it was strongly recommended by Māori and Pacific team members as a way to enhance recruitment of Māori and Pacific participants. The fact that ‘word of mouth’ was the second most effective strategy is an indication of the potential effectiveness of such face-to-face community-based recruitment. However, as this trial recruited people from all across NZ in-person recruitment was considered to be too logistically challenging and resource intensive. Furthermore, the potential for in-person recruitment to introduce selection bias was considered a risk to the internal validity of the study since those recruited using face-to-face methods may differ in other important ways from those recruited using alternative strategies as has been observed in other studies.28 For research to be truly representative of Māori and Pacific peoples, it is clear from this work that in future research, recruitment protocols and indeed study design need to be carefully planned and adapted to accommodate different cultural perspectives.

Studies focused on recruitment of underserved or hard-to-reach populations consistently report that greater resources, more time and targeted strategies are needed to recruit such populations.29 On the other hand, recruitment targets for other ethnicities could potentially be achieved with substantially less cost, thus freeing up more time and resources to focus on more priority population recruitment. Facebook recruitment alone achieved 88% (n=440) of the target for non-Māori and non-Pacific participants, almost eliminating the need to use any other recruitment strategies for this group.

To our knowledge, this is the first study to examine the success of a range of strategies to recruit participants to a smartphone-delivered study in NZ. Use of diagnostic technology, namely Google Analytics, enabled objective assessment of web-based recruitment strategies and their effectiveness. The current study also provides data on participant retention, which was previously identified as a gap among existing reports on recruitment for web-based and mobile health studies.7 We also describe strategies for reducing duplicate and fake registrations.

A limitation was the broad grouping used for self-reported recruitment sources, which prevented analysis at a more precise category level. Inability to estimate staffing costs associated with recruitment strategies was another limitation, likely to be particularly important in assessing research team network promotions, a heterogeneous approach that involved reasonably high resource for example, distribution of hard copy advertising fliers.

In conclusion, recruitment via the internet and social media is comparable in cost and substantially more effective than traditional study recruitment strategies such as mainstream media advertising, and it is effective in reaching diverse ethnic groups. However, additional targeted strategies should be considered where large number of participants from particular ethnic groups are sought.