Optimal survey design for community intervention evaluations: cohort or cross-sectional?

Community intervention evaluations that measure changes over time may conduct repeated cross-sectional surveys, follow a cohort of residents over time, or (often) use both designs. Each survey design has implications for precision and cost. To explore these issues, we assume that two waves of surveys are conducted, and that the goal is to estimate change in behavior for people who reside in the community at both times. Cohort designs are shown to provide more accurate estimates (in the sense of lower mean squared error) than cross-sectional estimates if (1) there is strong correlation over time in an individual's behavior at time 0 and time 1, (2) relatively few subjects are lost to followup, (3) the bias is relatively small, and (4) the available sample size is not too large. Otherwise, a repeated cross-sectional design is more efficient. We developed methods for choosing between the two designs, and applied them to actual survey data. Owing to drop-outs and losses to followup, the cohort estimates were usually more biased than the cross-sectional estimates. The correlations over time for most of the variables studied were also high. In many instances the cohort estimate, although biased, is preferred to the relatively unbiased cross-sectional estimate because the mean squared error was smaller for the cohort than for the cross-sectional estimate. If these results are replicated in other data, they may result in guidelines for choosing a more efficient study design.