The application of generalizability theory to surface electromyographic measurements during psychophysiological stress testing: how many measurements are needed?

Generalizability theory is an extension of classical reliability theory that allows multiple sources of measurement error in an experimental design to be investigated simultaneously. In the present study, generalizability theory was used to evaluate measurement error in psychophysiological test procedures used to differentiate tension headache patients from normal controls based upon measures of electromyographic (EMG) responding. Thirty-three subjects who met diagnostic criteria for tension-type headache and 40 normal control subjects who rarely or never experienced headache participated in two laboratory sessions. EMG activity of head and neck muscles was recorded while subjects performed baseline, relaxation, choice reaction time, psychomotor tracking, and cold pressor tasks. Variance components were computed for an experimental design having subjects nested within experimenters and crossed with sessions and replications. Generalizability coefficients were computed for combinations of various numbers of sessions and replications. The generalizability of EMG measures was highly variable, depending on the experimental conditions in force. The largest sources of measurement error were attributed to the unique responsiveness of individual subjects under a particular set of treatment conditions. For some stress tests currently in use, data from several testing sessions may need to be averaged in order to achieve acceptable levels of generalizability. Generalizability greater than 0.80 can be expected only rarely when data are collected during a single session. In the research setting, low generalizability may account for the failure of EMG-based stress tests to differentiate tension headache patients from controls during stressful task performance. In the clinical setting, the generalizability of information derived from "stress profiling" or muscle "scanning" techniques, which depend on results obtained during a single testing session, is doubtful.