Judging Relative Importance: Direct Rating and Point Allocation Are Not Equivalent

In this series of experiments we investigate two commonly used methods of assigning numerical values (i.e., decision weights) to attributes in order to signify their perceived relative importance. The two methods are to ask people to directly rate each of the attributes in turn (Rating), or to allocate a budget of points (typically 100 points) to the attributes (Point Allocation or PA). These procedures may seem to be minor variants of one another, yet they produce very different profiles of decision weights. The differences are predicted by a simple, idealized model of weighting, from which Rating and PA, in different ways, exhibit consistent elicitation-dependent bias.