Evaluation: salvation or nemesis of medical informatics?

The currently prevailing paradigms of evaluation in medical/health informatics are reviewed. Some problems with application of the objectivist approach to the evaluation of real-rather than simulated-(health) information systems are identified. The rigorous application of the objectivist approach, which was developed for laboratory experiments, is difficult to adapt to the evaluation of information systems in a practical real-world environment because such systems tend to be complex, changing rapidly over time, and often existing in a variety of variants. Practical and epistemological reasons for the consequent shortcomings of the objectivist approach are detailed. It is argued that insistence on the application of the objectivist principles to real information systems may hamper rather than advance insights and progress because of this. Alternatives in the form of the subjectivist approach and extensions to both the objectivist and subjectivist approaches that circumvent the identified problems are summarized. The need to include systems engineering approaches in, and to further extend, the evaluation methodology is pointed out.