A Markov model description of changeover probabilities on concurrent variable-interval schedules.

The primary data were peck-by-peck sequential records of four pigeons responding on several different concurrent variable-interval schedules. According to the hypothesis that the subject chooses the alternative with the highest probability of reinforcement at the moment, response-by-response performance in concurrent schedules should show sequential dependencies. However, such dependencies were not found, and it was possible to describe molecular-level performance with simple Markov chain models. The Markov model description implies that the momentary changeover probabilities were proportional to the overall relative reinforcement frequencies, and that changeover probabilities did not change as a function of previous responding. A second finding was that although a changeover-delay procedure was omitted, relative response frequencies closely approximated relative reinforcement frequencies.