Analysis of 2 x 2 tables of frequencies: matching test to experimental design.

BACKGROUND: Biomedical investigators often use unsuitable statistical techniques for analysing the 2 x 2 tables that result from their experimental observations. This is because they are confused by the conflicting, and sometimes inaccurate, advice they receive from statistical texts or statistical consultants.
METHODS: These consist of a review of published work, and the use of five different statistical procedures to analyse a 2 x 2 table, executed by StatXact 8.0, Testimate 6.0, Stata 10.0, SAS 9.1 and SPSS 16.0. Discussion and Conclusions It is essential to classify a 2 x 2 table before embarking on its analysis. A useful classification is into (i) Independence trials (doubly conditioned). These almost never occur in biomedical research because they involve predetermining the column and row totals in a 2 x 2 table. The Fisher exact test is the best method for analysing these trials. (ii) Comparative trials (singly conditioned). These correspond to the usual experimental design in biomedical work, in which a sample of convenience is randomized into two treatment groups, so that the group (column) totals are fixed in advance. The proper tests of significance are exact tests on the odds ratio, on the ratio of proportions (relative risk and risk ratio) or on the difference between proportions. (iii) Double dichotomy trials (unconditional). In these, a genuine random sample is taken from a defined population. Thus, neither column nor row totals are fixed in advance. The only practicable test is Pearson's chi(2)-test. In analysing any of the above trials, exact tests are to be much preferred to asymptotic (approximate) tests. The different commercial software packages use different algorithms for exact tests, and can give different outcomes in terms of P-values and confidence intervals. The most useful are StatXact and Testimate.