Corrected analysis of 'Using financial incentives to promote physical activity in American Indian adolescents: A randomized controlled trial' confirms conclusions.

In a recent article, Short et al.[1] presented results from an interesting experiment on how financial incentives can potentially increase physical activity among American Indian adolescents, particularly those from the Choctaw Nation of Oklahoma. Adolescents were to be randomly assigned to one of two treatment groups; however, in the case of siblings, the authors made an exception to the randomization by assigning all siblings within a sibship to the same group. While we understand the desire to minimize potential contamination between siblings, this is a violation of the planned individual randomization. Rather, randomization occurred at the level of the family, and it should not be assumed that the siblings are independent subjects. Therefore, it is incorrect to analyze them as independent observations in a regression model [2] as Short et al initially did. Options are either to have only included one sibling from each family, or to account for this correlation in analysis, such as one does in cluster-randomized trials [3,4].

This issue of correlated data is common in many randomized experiments, including students within schools [5] and animals within litters [6-8]. A simplistic way to analyze such cases could be to either only include one observation from each cluster or use the mean of the cluster as the value for analysis, but a better is to include a random effect for the cluster. This way, one can retain all the information and account for the variability within and among clusters.

For this paper, Ms. Golzarri-Arroyo and Dr. Allison contacted the original authors who graciously supplied the raw data including identifiers for siblings (13 pairs) and did so in a timely fashion. We are also grateful for their prompt responses to our questions and for engaging in dialogue about their project.

To first verify the reproducibility of their results, we replicated the published analyses using t-tests and were able to obtain the original results with negligible discrepancies in Means and SD. We also noticed a typographical error in the second to last footnote of Table 1, which should read “Increase from baseline to end of Phase 1 within group, p<0.05”. The error was noted and a correction has since been issued.

Table 1

P-values for difference between groups.

	Phase 1		Phase 2
Outcomes	Short et al.	Golzarri-Arroyo et al.	Short et al.	Golzarri-Arroyo et al.
	t-test	Linear Mixed Model, random effect for family	t-test	Linear Mixed Model, random effect for family
BMI, kg/m2	0.431	0.336	0.179	0.269
BMI, percentile	0.492	0.431	0.447	0.681
Body fat, %	0.823	0.780	0.990	0.857
Fat-free mass, kg	0.448	0.448	0.198	0.216
Waist circumference, cm	0.478	0.428	0.309	0.433
VO2peak, ml/kg FFM/min	0.210	0.251	0.967	0.841
Steps per day	0.848	0.941	0.262	0.354
Glucose, mmol/l	0.726	0.749	0.510	0.335
Insulin, pmol/l	0.965	0.796	0.436	0.593
iHOMA2 (%S)	0.228	0.297	0.845	0.986
HbA1c (%)	0.681	0.794	0.968	0.918
Exercise sessions performed	0.568	0.494	0.469	0.355
Total MVPA time, h	0.942	0.873	0.096	0.102
MVPA time per exercise session, minutes	0.131	0.151	0.002*	0.012*
MVPA time in moderate intensity range, %	0.680	0.716	0.817	0.653
Payments for exercise, USD$	<0.001*	<0.001*	0.043*	0.044*

* p < 0.05

We then performed corrected analyses by performing separate linear mixed models on each outcome. The mixed models included a random intercept for each family in order to account for clustering of outcomes among the 13 sib-pairs. In phase 1 (first 16 weeks) there were 6 sibling pairs in the standard payment group and 7 sibling pairs in the incentive payment group. In phase 2 (second 16 weeks) there were 4 and 5 sibling pairs in the standard and incentive payment groups, respectively. The number of participants was lower in phase 2 than phase 1 due to withdrawal from the study.

Fortunately, our results support the original conclusions and future readers can be assured by these results. The p-values from the results of both analyses are shown in Table 1, where the conclusions of significant results remain the same. There was a significant difference between groups on payments for exercise in both the original and revised analyses, where the treatment group received more payments than the control group in phase 1 (p<0.001 both analyses) and phase 2 (p = 0.043 originally, p = 0.044 revised). Also, in Phase 2, the treatment group had longer MVPA time per exercise sessions compared to the control group (p = 0.002 originally, p = 0.012 revised).

(PDF)

Click here for additional data file.