Gotta catch'em all! Pokémon GO and physical activity among young adults: difference in differences study.

OBJECTIVE: To estimate the effect of playing Pokémon GO on the number of steps taken daily up to six weeks after installation of the game.
DESIGN: Cohort study using online survey data. PARTICIPANTS: Survey participants of Amazon Mechanical Turk (n=1182) residing in the United States, aged 18 to 35 years and using iPhone 6 series smartphones. MAIN OUTCOME MEASURES: Number of daily steps taken each of the four weeks before and six weeks after installation of Pokémon GO, automatically recorded in the "Health" application of the iPhone 6 series smartphones and reported by the participants. A difference in difference regression model was used to estimate the change in daily steps in players of Pokémon GO compared with non-players.
RESULTS: 560 (47.4%) of the survey participants reported playing Pokémon GO and walked on average 4256 steps (SD 2697) each day in the four weeks before installation of the game. The difference in difference analysis showed that the daily average steps for Pokémon GO players during the first week of installation increased by 955 additional steps (95% confidence interval 697 to 1213), and then this increase gradually attenuated over the subsequent five weeks. By the sixth week after installation, the number of daily steps had gone back to pre-installation levels. No significant effect modification of Pokémon GO was found by sex, age, race group, bodyweight status, urbanity, or walkability of the area of residence.
CONCLUSIONS: Pokémon GO was associated with an increase in the daily number of steps after installation of the game. The association was, however, moderate and no longer observed after six weeks. Published by the BMJ Publishing Group Limited. For permission to use (where not already granted under a licence) please go to http://group.bmj.com/group/rights-licensing/permissions.

Introduction

Pokémon GO is an augmented reality game in which players search real world locations for cartoon characters appearing on their smartphone screen. Since its launch in July 2016, the game has been downloaded over 500 million times worldwide.

Games that incentivise exercise might have the potential to promote and sustain physical activity habits.1 2 Because walking is encouraged while playing, Pokémon GO has been suggested to increase physical activity and improve public health, but these claims have been based on anecdotal evidence.3 4 5

We used an online survey and automatically recorded step count data from iPhone devices to estimate the change in daily steps after installation of Pokémon GO among young adults in the United States.

Methods

Study population

We conducted an online survey using the Amazon Mechanical Turk (MTurk), between 1 and 31 of August 2016. MTurk is a website for recruitment of online workers (>500 000 registered workers worldwide6) who receive a small compensation for completing tasks, including responding to surveys. It is widely used for research purposes: it allows for fast recruitment of diverse study populations, and the reliability of MTurk is similar to that of traditional survey methods.7 8 9

We recruited survey participants who were aged 18 to 35 years, resided in the US, and used an iPhone 6 series smartphone, because these devices automatically record the number of steps taken while carrying the device. Participants were paid $2.0 (£1.6; €1.9) for completed study participation. Because the number of people fulfilling the inclusion criteria who were exposed to the online posting of our study is not known, it was not possible to calculate the response rate. Of the 2225 people who responded to the survey, we excluded 115 who had downloaded Pokémon GO but not achieved a “trainer level” of 5 or more, which is reached after around two hours of walking. At this level, functions integral to the game are unlocked, including joining a team and competing for Pokémon gyms. We further excluded 67 people who did not complete the survey, and 861 who did not provide steps data through screenshots (picture of the screen content). The characteristics of these individuals were similar to the 1182 individuals who provided all information required for study participation (see supplementary table 1).

Covariates

Survey participants were asked about their zip code and urbanity of residence, education level, household income, marital status, and weight and height from which body mass index was calculated. Participants who reported that they played Pokémon GO were asked to upload a screenshot of their Pokémon GO application, which shows the installation date. We matched the zip code of residence with a validated neighborhood web based walkability index (Walk score), which was categorised into four groups (car dependent, somewhat walkable, very walkable, and walker’s paradise).10 11

Participants were asked to upload screenshots showing their number of steps daily while carrying their iPhone, for each day during the two months preceding the date of study participation. This information is recorded by default in the “Health” application of iPhone 6 series, using the device’s built-in accelerometer.

Statistical analyses

We compared the average number of steps taken each day for each of the four weeks before installation of the game compared with the steps during each of the six weeks after installation in players of Pokémon GO and non-players, respectively. For non-players, we compared the number of steps before and after the median date of installation among playing participants. To examine the effect of Pokémon GO on daily steps, we conducted a difference in difference analysis. This method attempts to mimic an experimental research design by comparing the change in the outcome over time in the treated group compared with the control group, under the assumption that the differences between the groups would have remained constant under no treatment. We used a multivariate regression model; the estimate of change in number of steps after Pokémon GO was obtained from interaction indicators for players and each week after installation of the game. The model also adjusted for time invariant differences between players and non-players and for fluctuations in steps across weeks before and after installation. We used the Stata 1212 regress command, and we calculated robust standard errors (cluster subcommand) to account for the repeated measurements within the same individuals.

In separate models, we assessed effect modification of Pokémon GO by sex, age group (18-24 and 25-35 years), annual household income (<$50 000 and ≥$50 000), race (Asian, white, black, and other), bodyweight status (body mass index <25 and ≥25), urbanity (urban, suburban, or rural), and walkability index group (car dependent or somewhat walkable, and very walkable or walker’s paradise). To the main model we added the variable, and interaction terms between the variable, Pokémon GO and week since installation of the game (difference in difference in difference13). Finally, we adjusted for differences in time trends of daily steps across baseline variables, the distribution of which might differ between players and non-players; to the main model, we added baseline variables, and interaction variables between baseline variables and each week after installation of the game.

Results

Of the survey participants, 560 (47.4%) played Pokémon GO at a trainer level of 5 or more. Of the players, 90% (n=622) installed the game within 10 days of its release on 6 July 2016 (median 8 July 2016). Playing Pokémon GO was common across various subgroups of the population (table 1); players, however, tended to be younger, have a lower education and household income, and be obese, and were more likely to be single and less likely to be black compared with non-players (table 1).

Table 1

Characteristics of Pokémon GO* players and non-players. Values are numbers (percentages) unless stated otherwise

Characteristic	Non-players (n=622)	Players (n=560)
Mean (SD) age (years)	27.3 (4.5)	25.6 (4.3)
Age group (years):
18-24	188 (30.2)	242 (43.2)
25-35	434 (69.8)	318 (56.8)
Women	453 (72.8)	390 (69.6)
Race:
Asian	32 (5.1)	38 (6.8)
White	491 (78.9)	430 (76.8)
Black	68 (10.9)	36 (6.4)
Other	31 (5.0)	56 (10.0)
Education:
High school or less	29 (4.7)	51 (9.1)
Some university	208 (33.5)	237 (42.5)
College degree	267 (43.0)	207 (37.1)
Graduate/professional degree	117 (18.8)	63 (11.3)
Marital status:
Single	336 (54)	365 (65.2)
Married/cohabitating	256 (41.2)	174 (31.1)
Divorced/separated/widowed	30 (4.8)	21 (3.8)
Household income:
<50 000	240 (38.6)	277 (49.5)
50 000 to <100 000	269 (43.3)	189 (33.8)
≥100 000	113 (18.2)	94 (16.8)
Region:
Midwest	187 (30.1)	175 (31.3)
Northeast	87 (14.0)	74 (13.2)
South	157 (25.2)	128 (22.9)
West	191 (30.7)	183 (32.7)
Urbanity:
Rural	99 (15.9)	77 (13.8)
Suburb	294 (47.3)	290 (51.8)
Urban	229 (36.8)	193 (34.5)
Walk score category:
Car dependent	431 (69.7)	391 (70.0)
Somewhat walkable	69 (11.2)	71 (12.7)
Very walkable	63 (10.2)	52 (9.3)
Walker’s paradise	55 (8.9)	45 (8.1)
Mean (SD) body mass index	26.9 (6.7)	27.9 (7.7)
Bodyweight status (BMI):
Normal (<25)	295 (47.4)	246 (44.0)
Overweight (25 to <30)	165 (26.5)	130 (23.3)
Obese (≥30)	162 (26.1)	183 (32.7)
Mean (SD) daily steps in four weeks before installation of game	4127 (2930)	4257 (2698)

*Installation of Pokémon GO: median 8 July (interquartile range 7-11 July).

In the four weeks before installation of Pokémon GO, participants who played the game took on average 4256 (SD 2697) steps daily. The corresponding number for non-players in the four weeks preceding 8 July (median date of Pokémon GO installation among the players) was 4126 (SD 2930). After installation of the game, the daily steps among players increased sharply before gradually returning to the pre-installation levels, whereas the number of daily steps for non-players remained at similar levels throughout the study period. Among players, daily steps increased to 5123 (SD 3371) during the first week after installation and was 4994 (3374) in week 2, 4693 (3104) in week 3, 4499 (3077) in week 4, 4108 (2927) in week 5, and 3985 (2888) in week 6 (fig 1). The difference in difference analysis confirmed the pattern: Pokémon GO was associated with an increase in daily steps of 955 (95% confidence interval 697 to 1213) during the first week, the effect was gradually attenuated over the subsequent weeks, and by week 6 it was not significant (130, −593 to 853) (fig 2 and supplementary table 2).

Fig 1 Average number of daily steps and 95% confidence intervals by week before and after installation of Pokémon GO (median 8 July 2016)

Fig 2 Average daily number of steps before and after installation of Pokémon GO (median 8 July 2016). Confidence intervals are estimated using a difference in difference regression model (see supplementary table 1 for the full model)

The analyses on effect modification of Pokémon GO by baseline variables showed no significant interaction between the variables, Pokémon GO and week since installation of the game (all P values ≥0.076, see supplementary table 3). Adjusting for differences in time trends in number of steps by baseline variables did not materially affect the results (see supplementary table 4).

Discussion

In this study on the impact of Pokémon GO on physical activity in young US adults, we found that in the first week after installation the game was associated with an increase in daily number of steps of 955 (95% confidence interval 697 to 1213). The association was attenuated during the subsequent weeks and was no longer significant by the sixth week. Assuming steps of 0.8 m at a pace of 4 km/h,14 the change in number of steps in the first week would translate into 11 minutes of additional walking daily, which is around half of the World Health Organization recommendation of 150 or more minutes weekly.15

Pokémon GO has been suggested to improve public health by promoting physical activity.3 4 5 In our study population, however, the results indicate that the health impact of Pokémon GO might be moderate. Interventions designed to increase walking typically increase the number of steps by 2500 daily.16 Even if smaller amounts of physical activity might also be important for health outcomes,15 17 the increase in steps from Pokémon GO, as with many physical activity interventions,18 was not sustained over time. Pokémon GO might also entail risks, such as injuries and road traffic incidents.19 20

Strengths of this study include the use of automatically recorded data on steps from the iPhone, which has been shown to accurately track step counts21; validation of Pokémon GO installation and date using screenshots; and a study population from a wide range of sociodemographic groups and regions of the US.

Our study has limitations. Firstly, steps were only recorded when the iPhone was carried. This might lead to overestimation of the effect of Pokémon GO because physical activity could occur when the phone is not being carried (eg, football), whereas all Pokémon GO related activity was recorded. Secondly, although our results did not change materially when adjusting for several baseline variables, the difference in difference analysis is susceptible to confounding from changes in number of daily steps that were unique to Pokémon GO players but independent of the game. Thirdly, our study population was limited to iPhone users in the US, and MTurk workers are not representative of the general population—for example, in terms of sex (overrepresentation of women), political views, and personality.9 22 Generalisability of our results might therefore be limited.

Although the association between Pokémon GO and change in number of steps was short lived in our study, some people might sustain increased physical activity through the game. Also, the effect of Pokémon GO on physical activity might be different in children, who were not included in our study. Other potential benefits might exist, such as increased social connectedness and improved mood.

Pokémon GO is an augmented reality game overlaying graphics on the real world using the camera function of smartphones

The game has been downloaded over 500 million times worldwide

Because the game encourages walking it has been suggested to increase physical activity

In young US adults using iPhones, Pokémon GO was associated with a moderate increase in daily number of steps

This increase was not, however, sustained over time: the number of steps had gone back to pre-installation levels by the sixth week after installation of the game

The impact of Pokémon GO on physical activity was limited in our study population of young adults in the US