Separating model optimization and model validation in statistical cross-validation as applied to crystallography.

Statistical cross-validation has become an integral part of the model-refinement process in macromolecular crystallography. However, the test set of reflections, for which the free R value is calculated, is used both to optimize the parameterization of the structure model and to validate the model itself. This practice could introduce bias and diminish the value of R(free) as an independent check of model quality. It is proposed here that by introducing a dormant hold-out set of reflections, any problems with such bias can be avoided. This procedure requires only a small modification of the standard cross-validation protocol.