General Gittins index processes in discrete time.

We combine the formulation of Mandelbaum [Mandelbaum, A. (1986) Probab. Theory Rel. Fields 71, 129-147] with ideas from Whittle [Whittle, P. (1980) J. R. Stat. Soc. B 42, 143-149] to obtain a simple and constructive proof for the optimality of Gittins index processes in the general, nonmarkovian dynamic allocation (or "multi-armed bandit") problem. Our approach also provides an explicit expression for the value of this problem.