Does speed in completing an online questionnaire have an influence on its reliability?

A common way to attract Internet users to complete an online questionnaire is to offer the opportunity to win money in a lottery. If participants hurry through the questionnaire to head for a reward, Cronbach's alpha, an indicator for internal consistency and therefore reliability of a questionnaire, could be affected, because these participants probably do not read the items carefully but instead click randomly through the questionnaire. We developed a system to measure the total time needed to complete the questionnaire in order to identify different groups of participants according to differences in fill-in time (speed). Five hundred seventy-five respondents participated in an online study testing the reliability of the Affective Neuroscience Personality Scales (ANPS). Results showed that speed has no influence on Cronbach's alpha with the exception of a lie scale, which consisted in contrast to the personality scales (12-14 items) only of five items. Only 2% of all participants were suspected to be cheaters with respect to the time spent on answering 110 items of the ANPS. Interestingly, Cronbach's alpha remained stable in this extremely fast group, which spent less than 2 SD below mean-completion time on the questionnaire.