OBJECTIVE: This report examines the impact of collecting and analyzing sequential data from the same sample using the chi-square test. Researchers in rehabilitation frequently analyze categorical data collected repeatedly from the same sample. DESIGN: Simulations were conducted using the chi-square test to analyze data with correlations between pretest and posttest ranging from .20 to .80. The simulations were analyzed using the standard chi-square based on a 2 x 2 contingency table. RESULTS: An incidence of type 1 errors of approximately .30 for an expected alpha of .05 was found when the correlation (serial dependence) between pretest and posttest was high (phi = .80). For simulations using an alpha = .01, the type 1 error rate was .18 for data with substantial correlation (phi = .80). CONCLUSION: Rehabilitation researchers who use the chi-square test with data collected repeatedly from the same sample risk committing a type 1 error and misinterpreting their results. Alternatives to the standard chi-square test, such as McNemar's test, should be considered when correlated data are analyzed using a 2 x 2 contingency table.
OBJECTIVE: This report examines the impact of collecting and analyzing sequential data from the same sample using the chi-square test. Researchers in rehabilitation frequently analyze categorical data collected repeatedly from the same sample. DESIGN: Simulations were conducted using the chi-square test to analyze data with correlations between pretest and posttest ranging from .20 to .80. The simulations were analyzed using the standard chi-square based on a 2 x 2 contingency table. RESULTS: An incidence of type 1 errors of approximately .30 for an expected alpha of .05 was found when the correlation (serial dependence) between pretest and posttest was high (phi = .80). For simulations using an alpha = .01, the type 1 error rate was .18 for data with substantial correlation (phi = .80). CONCLUSION: Rehabilitation researchers who use the chi-square test with data collected repeatedly from the same sample risk committing a type 1 error and misinterpreting their results. Alternatives to the standard chi-square test, such as McNemar's test, should be considered when correlated data are analyzed using a 2 x 2 contingency table.