Brief Report: Negative Controls to Detect Selection Bias and Measurement Bias in Epidemiologic Studies.

Biomedical laboratory experiments routinely use negative controls to identify possible sources of bias, but epidemiologic studies have infrequently used this type of control in their design or measurement approach. Recently, epidemiologists proposed the routine use of negative controls in observational studies and defined the structure of negative controls to detect bias due to unmeasured confounding. We extend this previous study and define the structure of negative controls to detect selection bias and measurement bias in both observational studies and randomized trials. We illustrate the strengths and limitations of negative controls in this context using examples from the epidemiologic literature. Given their demonstrated utility and broad generalizability, the routine use of prespecified negative controls will strengthen the evidence from epidemiologic studies.

Negative controls are used in laboratory science to help detect problems with the experimental method. In epidemiologic studies, a negative control outcome acts as a surrogate for the actual outcome—the negative control should be subject to the same potential sources of bias as the outcome but is not caused by the exposure of interest. Negative control exposures are conceptually the same, but defined relative to the actual exposure. Lipsitch et al.[1] defined the structure of negative controls to detect unmeasured confounding and described by way of example how negative controls could be used to detect selection bias and measurement (information) bias. Here, we define the causal structure of negative controls with respect to selection bias[2] and measurement bias,[3] and illustrate their use with published examples.

NEGATIVE CONTROLS TO DETECT SELECTION BIAS

For clarity, we have focused on structures of selection bias under the null (no effect of exposure) and have focused on four structures we would expect to be most relevant to epidemiologic research (Fig. 1).[2] Selection bias occurs when the analysis conditions on a third variable C that is a common descendant of exposure A and outcome Y or a common descendant of unmeasured causes of either A or Y or both, denoted U or U.[2] Defining C as the combination of censoring mechanisms during enrollment, follow-up, and analysis, standard epidemiologic measures are limited to the stratum of C = 0 (uncensored, available data). We denote negative control exposures as N and negative control outcomes as N—they could be dichotomous, categorical, or continuous.

FIGURE 1.

Simplified causal diagrams of selection bias for exposure A and outcome Y along with negative control exposures (N) and outcomes (N). In all four structures, selection bias results from conditioning on C, a common descendant of (A) exposure A and outcome Y, (B) cause of exposure U and outcome Y, (C) exposure A and cause of outcome U, or (D) cause of exposure U and cause of outcome U.

A common form of selection bias can result from conditioning on a common descendant of the exposure and outcome (Fig. 1A). For example, in case–control designs where selection into the study (C) conditions on the outcome (Y → C), selection bias results if the exposure affects participant selection (A → C) differentially by case/control status. This bias structure could also occur in the re-analysis of a case–control study for a secondary outcome Z, which is intermediate between the exposure and outcome: A → Z → Y → C. This design is used in genetic epidemiology studies that repurpose costly genomic measures A and look at their association with additional outcomes Z.[4,5] Negative control outcomes or exposures to detect this type of bias would need to similarly affect participant selection (N → C, Table, example 1 or N → C, Table, example 2).

FIGURE 2.

Simplified causal diagrams of differential measurement error for an exposure A that causes outcome Y. The basic structures for outcome measurement error (A) and exposure measurement error (B) are summarized along with negative control exposures (N) and outcomes (N). U represents other causes of the measured value of Y* and U represents other causes of the measured value of A*.

A second form of selection bias can occur in cross-sectional or retrospective studies when the outcome Y and an unmeasured cause of the exposure U affect study enrollment C (Fig. 1B). This bias can be detected by using a negative control exposure that shares the same unmeasured parent of the exposure (U → N, Table, example 3) or a negative control outcome that similarly affects enrollment (N → C).

A third form of selection bias can occur when a study conditions on a common descendant of the exposure and an unmeasured cause of the outcome (Fig. 1C). In per-protocol analyses of randomized trials, investigators limit the analysis to individuals that complied with their respective group assignments. Bias results if compliance (C) is determined by treatment assignment (A) and by unmeasured characteristics (U) that affect both individuals’ willingness to comply with their assigned treatment and their outcome.[6] For example, if individuals assigned to treatment who comply with their regimen are more health conscious than noncompliers, a naive per-protocol analysis could overestimate the benefits of the treatment. Figure 1C also applies to selection bias in prospective studies if exposure A and an unmeasured cause of the outcome U affect loss to follow-up C. A negative control outcome that shares the same unmeasured parent as the outcome (U → N, Table, example 4) or a negative control exposure that similarly affects enrollment, loss to follow-up or compliance (N → C) can be used to detect this type of selection bias.

Finally, selection bias can occur if a cause of the exposure U and a cause of the outcome U both affect enrollment C (Fig. 1D). One example is volunteer bias in cohort studies,[2] where individuals’ underlying characteristics might affect their exposures and health outcomes as well as their decision to enroll in the study. This bias could be detected by using a negative control outcome that shares the same parent as the actual outcome (U → N) or a negative control exposure that shares the same parent as the actual exposure (U → N); however, we are unaware of a study that has used negative controls for this bias structure.

NEGATIVE CONTROLS TO DETECT MEASUREMENT BIAS

Many studies have measurement error so that investigators observe Y*, which is an error-prone version of the outcome Y.[3] For example, if Y is an enteric infection that causes diarrhea, Y* could be caregiver-reported diarrhea symptoms. In the diagrams, we assume U accounts for all other unmeasured causes of Y* beyond Y. Similarly, A* can be subject to unmeasured sources of error U. We focus our definitions on differential measurement errors (Fig. 2) because they are most likely to cause bias and the consequent bias is often the least predictable.[3] For parsimony, we have not provided formal definitions of negative controls under more complex (or simple nondifferential) measurement error scenarios, but in principle Figure 2 could be extended to accommodate them—for example, removing edges Y → U or A → U defines negative controls for independent, nondifferential errors.

Differential outcome measurement error occurs when A influences the measured outcome Y* through U (Fig. 2A). In an unblinded study, physician follow-up (U) may be increased in treated patients compared with the untreated (A → U), and selective follow-up causes differential measurement error of Y. Differential outcome reporting can also bias observational studies and unblinded trials with subjectively reported outcomes,[7] where participant knowledge of their exposure or treatment assignment could influence reporting. An ideal negative control outcome for this scenario shares a common source of correlated measurement error (U) with the true outcome (Table, example 5). Negative control exposures for differential outcome measurement error also exist—placebo drugs in clinical trials are a classic example. Negative control exposures that act like a placebo can be devised for observational studies (Table, example 6).

Differential exposure measurement error is possible when the exposure A is measured concurrently with or after the occurrence of the outcome Y (Fig. 2B) and is of greatest concern in retrospective or cross-sectional studies. For example, retrospective case–control studies can be biased if they rely on self-reported exposures A* as a proxy for true exposures A and cases remember exposures more accurately than controls. Negative controls for exposure measurement error need to share correlated errors (U) with the exposure (Table, example 7). The bias described in Figure 2B can also occur if an outcome is measured concurrently with an exposure, where the measured exposure (A*) is used as a proxy for the same measure at a time in the past that is relevant for causing disease (A) (Table, example 8).[8-15]

DISCUSSION

We defined the structure of negative controls to detect common forms of selection and measurement bias in observational studies and randomized trials. The examples in the Table illustrate many recent applications, and the structural definitions in Figures 1 and 2 generalize to further applications we have not discussed—for example, Figure 1C describes the structure for healthy worker bias[2] and healthy user/adherer bias.[16] For extensions beyond the detection of bias, recent efforts have used negative controls in sensitivity analyses to quantify the magnitude of bias from unobserved confounding,[17] as a tool to remove bias in standardized mortality ratios,[18] and as a basis for large-scale empirical calibration of P values in drug safety studies.[19] We envision similar extensions for the types of negative controls defined here.

Negative controls have some limitations that arise in practice. Lipsitch et al.[1] characterized negative controls as a “blunt tool” to detect bias in the context of confounding, and that characterization is equally apt in the context of selection and measurement bias. Negative controls often lack specificity in the type of bias that they detect—many examples in the Table illustrate this limitation (Discussion in the eAppendix, http://links.lww.com/EDE/B56). Moreover, negative controls may identify the presence of bias but cannot in general determine its direction or magnitude without additional assumptions.[1] Another limitation that many negative controls share is that they often fail to provide a definitive test of the absence of bias.[1],20 All of these limitations coalesce into a common challenge for selecting negative controls: a control must meet its assumed structural definition, otherwise it can be an insensitive or inappropriate diagnostic for bias. Thus, the ability of a negative control to adequately detect bias ultimately relies on the plausibility of (often untestable) assumptions encoded in its causal diagram. Finally, prespecification of primary outcome and exposure definitions helps prevent the selective presentation of favorable results, and prespecification and complete reporting of negative controls would prevent similar problems.20

Selection bias or measurement bias threaten nearly every epidemiologic study design. Given their demonstrated utility and broad generalizability, the routine use of negative controls will help detect selection bias and measurement bias in epidemiologic studies.

Table.

Examples of Studies that Have Used Negative Controls to Detect Selection or Measurement Bias Following Bias Structures in Figures 1 and 2