Best-classifier feedback in diagnostic classification training.

Diagnostic classification training requires viewing many examples along with category membership feedback. "Objective" feedback based on category membership suggests that perfect accuracy is attainable when it may not be (e.g., with confusable categories). Previous work shows that feedback based on an "optimal" responder (that sometimes makes classification errors) leads to higher long-run reward, especially in unequal category payoff conditions. In the current study, participants learned to classify normal or cancerous mammography images, earning more points for correct "cancer" than "normal" responses. Feedback was either objective or based on performance of an empirically determined "best" classifier. This approach is necessary because theoretically optimal responses cannot be determined with complex real-world stimuli with unknown perceptual distributions. Replicating earlier work that used simple artificial stimuli, we found that best-classifier performance led to decision-criterion values (β) closer to the reward-maximizing criterion, along with higher point totals and a slight reduction (as predicted) in overall accuracy.