| Literature DB >> 31436848 |
Martijn J Schuemie1,2,3, Patrick B Ryan1,2,4, Kenneth K C Man5,6,7,8, Ian C K Wong5,6, Marc A Suchard1,3,9,10, George Hripcsak1,4,11.
Abstract
The case-control design is widely used in retrospective database studies, often leading to spectacular findings. However, results of these studies often cannot be replicated, and the advantage of this design over others is questionable. To demonstrate the shortcomings of applications of this design, we replicate two published case-control studies. The first investigates isotretinoin and ulcerative colitis using a simple case-control design. The second focuses on dipeptidyl peptidase-4 inhibitors and acute pancreatitis, using a nested case-control design. We include large sets of negative control exposures (where the true odds ratio is believed to be 1) in both studies. Both replication studies produce effect size estimates consistent with the original studies, but also generate estimates for the negative control exposures showing substantial residual bias. In contrast, applying a self-controlled design to answer the same questions using the same data reveals far less bias. Although the case-control design in general is not at fault, its application in retrospective database studies, where all exposure and covariate data for the entire cohort are available, is unnecessary, as other alternatives such as cohort and self-controlled designs are available. Moreover, by focusing on cases and controls it opens the door to inappropriate comparisons between exposure groups, leading to confounding for which the design has few options to adjust for. We argue that this design should no longer be used in these types of data. At the very least, negative control exposures should be used to prove that the concerns raised here do not apply.Entities:
Keywords: case control; database studies; methods; retrospective studies
Year: 2019 PMID: 31436848 PMCID: PMC6771795 DOI: 10.1002/sim.8215
Source DB: PubMed Journal: Stat Med ISSN: 0277-6715 Impact factor: 2.373
Patient characteristics of exposed and unexposed in the Crockett replication. S.Diff: Standardized difference of means (exposed – unexposed)
|
|
|
|
|
|
|
|
|
|---|---|---|---|---|---|---|---|
| Medical history: General | Peripheral vascular disease | 0.6 | 2.4 | 0.11 | |||
| Acute respiratory disease | 16.6 | 21.9 | 0.09 | Pulmonary embolism | 0.0 | 0.2 | 0.04 |
| ADHD | 2.3 | 1.1 | −0.06 | Venous thrombosis | 0.4 | 0.8 | 0.04 |
| Chronic liver disease | 0.6 | 0.9 | 0.03 | Medical history: Neoplasms | |||
| Chronic obstructive lung disease | 0.2 | 1.2 | 0.08 | Hematologic neoplasm | 0.2 | 0.4 | 0.02 |
| Crohn's disease | 2.3 | 3.5 | 0.05 | Malignant lymphoma | 0.0 | 0.2 | 0.04 |
| Dementia | 0.2 | 0.1 | −0.02 | Malignant neoplasm of anorectum | 0.0 | 0.1 | 0.03 |
| Depressive disorder | 4.2 | 5.6 | 0.04 | Malignant neoplastic disease | 1.0 | 3.2 | 0.11 |
| Diabetes mellitus | 0.7 | 5.8 | 0.20 | Malignant tumor of breast | 0.1 | 0.7 | 0.07 |
| Gastroesophageal reflux disease | 1.9 | 4.9 | 0.11 | Malignant tumor of colon | 0.0 | 0.2 | 0.04 |
| Gastrointestinal hemorrhage | 1.6 | 4.0 | 0.10 | Malignant tumor of lung | 0.0 | 0.1 | 0.03 |
| HIV infection | 0.3 | 0.2 | −0.02 | Malignant tumor of urinary bladder | 0.0 | 0.1 | 0.03 |
| Hyperlipidemia | 3.3 | 16.0 | 0.29 | Prostate cancer | 0.0 | 0.3 | 0.05 |
| Hypertensive disorder | 2.4 | 15.5 | 0.31 | Medication use | |||
| Lesion of liver | 0.1 | 0.3 | 0.03 | Agents acting on the RAS | 1.6 | 11.6 | 0.27 |
| Obesity | 0.5 | 2.4 | 0.11 | Antibacterials for systemic use | 58.9 | 43.4 | −0.15 |
| Osteoarthritis | 1.9 | 6.8 | 0.17 | Antidepressants | 9.5 | 14.9 | 0.11 |
| Pneumonia | 0.8 | 1.2 | 0.03 | Antiepileptics | 2.8 | 5.3 | 0.09 |
| Psoriasis | 0.9 | 0.8 | −0.01 | Antiinflam. and antirheum. drugs | 8.7 | 16.0 | 0.15 |
| Renal impairment | 0.2 | 0.8 | 0.06 | Antineoplastic agents | 14.9 | 2.2 | −0.31 |
| Rheumatoid arthritis | 0.3 | 0.8 | 0.05 | Antipsoriatics | 5.5 | 0.5 | −0.21 |
| Schizophrenia | 0.0 | 0.1 | 0.02 | Antithrombotic agents | 0.8 | 2.5 | 0.10 |
| Ulcerative colitis | 0.0 | 0.0 | 0.02 | Beta blocking agents | 1.6 | 7.2 | 0.19 |
| Urinary tract infectious disease | 3.0 | 5.2 | 0.08 | Calcium channel blockers | 1.0 | 4.7 | 0.16 |
| Viral hepatitis C | 0.0 | 0.3 | 0.05 | Diuretics | 3.8 | 9.7 | 0.16 |
| Visual system disorder | 6.6 | 11.9 | 0.12 | Drugs for acid related disorders | 5.7 | 12.2 | 0.15 |
| Medical history: Cardiovascular disease | |||||||
| Atrial fibrillation | 0.1 | 0.6 | 0.06 | Drugs used in diabetes | 0.7 | 4.8 | 0.18 |
| Cerebrovascular disease | 0.3 | 0.7 | 0.04 | Immunosuppressants | 2.3 | 3.0 | 0.03 |
| Coronary arteriosclerosis | 0.1 | 1.9 | 0.13 | Lipid modifying agents | 1.5 | 12.4 | 0.29 |
| Heart disease | 1.5 | 6.4 | 0.17 | Opioids | 5.8 | 11.9 | 0.14 |
| Heart failure | 0.3 | 0.5 | 0.03 | Psycholeptics | 7.7 | 14.6 | 0.15 |
| Ischemic heart disease | 0.1 | 1.2 | 0.10 | Psychostimulants and nootropics | 3.3 | 2.7 | −0.02 |
Patient characteristics of exposed and unexposed in the Chou replication
|
|
|
|
|
|
|
|
|
|---|---|---|---|---|---|---|---|
| Medical history: General | Peripheral vascular disease | 6.4 | 7.9 | 0.04 | |||
| Acute respiratory disease | 24.8 | 24.0 | −0.01 | Pulmonary embolism | 0.2 | 0.4 | 0.02 |
| ADHD | 0.4 | 0.5 | 0.01 | Venous thrombosis | 1.4 | 1.6 | 0.02 |
| Chronic liver disease | 3.7 | 3.4 | −0.01 | Medical history: Neoplasms | |||
| Chronic obstructive lung disease | 2.4 | 3.6 | 0.05 | Hematologic neoplasm | 0.6 | 0.9 | 0.02 |
| Crohn's disease | 0.4 | 0.4 | 0.00 | Malignant lymphoma | 0.3 | 0.4 | 0.01 |
| Dementia | 0.1 | 0.2 | 0.01 | Malignant neoplasm of anorectum | 0.2 | 0.1 | 0.00 |
| Depressive disorder | 5.7 | 8.0 | 0.06 | Malignant neoplastic disease | 4.5 | 5.6 | 0.04 |
| Diabetes mellitus | 78.2 | 85.2 | 0.05 | Malignant tumor of breast | 0.9 | 1.0 | 0.01 |
| Gastroesophageal reflux disease | 7.8 | 8.3 | 0.01 | Malignant tumor of colon | 0.3 | 0.3 | 0.01 |
| Gastrointestinal hemorrhage | 2.3 | 2.4 | 0.00 | Malignant tumor of lung | 0.1 | 0.2 | 0.03 |
| HIV infection | 0.4 | 0.2 | −0.02 | Malignant tumor of urinary bladder | 0.0 | 0.2 | 0.03 |
| Hyperlipidemia | 46.2 | 50.4 | 0.04 | Prostate cancer | 0.5 | 0.8 | 0.02 |
| Hypertensive disorder | 49.8 | 54.8 | 0.05 | Medication use | |||
| Lesion of liver | 0.6 | 1.2 | 0.04 | Agents acting on the RAS | 53.1 | 57.6 | 0.04 |
| Obesity | 7.5 | 11.8 | 0.10 | Antibacterials for systemic use | 53.7 | 53.5 | 0.00 |
| Osteoarthritis | 12.3 | 14.6 | 0.05 | Antidepressants | 20.8 | 23.5 | 0.04 |
| Pneumonia | 1.9 | 2.6 | 0.03 | Antiepileptics | 10.0 | 11.8 | 0.04 |
| Psoriasis | 1.1 | 1.1 | 0.00 | Antiinflam. and antirheum. drugs | 25.7 | 27.1 | 0.02 |
| Renal impairment | 3.6 | 4.3 | 0.03 | Antineoplastic agents | 1.5 | 2.0 | 0.03 |
| Rheumatoid arthritis | 0.9 | 1.2 | 0.02 | Antipsoriatics | 0.5 | 0.6 | 0.01 |
| Schizophrenia | 0.1 | 0.2 | 0.02 | Antithrombotic agents | 7.9 | 9.9 | 0.05 |
| Ulcerative colitis | 0.3 | 0.4 | 0.01 | Beta blocking agents | 23.4 | 25.9 | 0.04 |
| Urinary tract infectious disease | 6.1 | 7.2 | 0.03 | Calcium channel blockers | 16.3 | 18.7 | 0.04 |
| Viral hepatitis C | 0.8 | 0.7 | 0.00 | Diuretics | 33.5 | 36.1 | 0.03 |
| Visual system disorder | 18.5 | 21.6 | 0.05 | Drugs for acid related disorders | 21.8 | 23.1 | 0.02 |
| Medical history: Cardiovascular disease | Drugs for obstructive airway diseases | 20.7 | 21.9 | −0.06 | |||
| Atrial fibrillation | 1.5 | 2.2 | 0.04 | Drugs used in diabetes | 64.2 | 86.1 | 0.18 |
| Cerebrovascular disease | 1.4 | 2.1 | 0.03 | Immunosuppressants | 2.0 | 2.5 | 0.02 |
| Coronary arteriosclerosis | 7.5 | 8.6 | 0.03 | Lipid modifying agents | 50.8 | 55.3 | 0.04 |
| Heart disease | 16.7 | 18.6 | 0.03 | Opioids | 19.9 | 20.8 | 0.01 |
| Heart failure | 2.5 | 2.9 | 0.02 | Psycholeptics | 19.9 | 22.6 | 0.04 |
| Ischemic heart disease | 4.3 | 4.9 | 0.02 | Psychostimulants and nootropics | 2.6 | 2.6 | 0.00 |
Figure 2Estimates from the self‐controlled case series (SCCS) design for the exposures of interest (yellow diamond) and the negative control exposures (blue dots) for the Crockett replication (left) and the Chou replication (right). Estimates below the dashed line have p < 0.05. Estimates in the orange area have a calibrated p < 0.05 [Colour figure can be viewed at wileyonlinelibrary.com]
Figure 1Estimates from the original study (purple triangle), our replication (yellow diamond), and our replication applied to the negative control exposures (blue dots) for the Crockett replication (left) and the Chou replication (right). Estimates below the dashed line have p < 0.05. Estimates in the orange area have a calibrated p < 0.05 [Colour figure can be viewed at wileyonlinelibrary.com]