Unbiasedness and efficiency of non-parametric and UMVUE estimators of the probabilistic index and related statistics.

In reliability theory, diagnostic accuracy, and clinical trials, the quantity P(X>Y)+1/2P(X=Y), also known as the Probabilistic Index (PI), is a common treatment effect measure when comparing two groups of observations. The quantity P(X>Y)-P(Y>X), a linear transformation of PI known as the net benefit, has also been advocated as an intuitively appealing treatment effect measure. Parametric estimation of PI has received a lot of attention in the past 40 years, with the formulation of the Uniformly Minimum-Variance Unbiased Estimator (UMVUE) for many distributions. However, the non-parametric Mann-Whitney estimator of the PI is also known to be UMVUE in some situations. To understand this seeming contradiction, in this paper a systematic comparison is performed between the non-parametric estimator for the PI and parametric UMVUE estimators in various settings. We show that the Mann-Whitney estimator is always an unbiased estimator of the PI with univariate, completely observed data, while the parametric UMVUE is not when the distribution is misspecified. Additionally, the Mann-Whitney estimator is the UMVUE when observations belong to an unrestricted family. When observations come from a more restrictive family of distributions, the loss in efficiency for the non-parametric estimator is limited in realistic clinical scenarios. In conclusion, the Mann-Whitney estimator is simple to use and is a reliable estimator for the PI and net benefit in realistic clinical scenarios.