Neural Network Exploration Using Optimal Experiment Design.

I consider the question "How should one act when the only goal is to learn as much as possible?". Building on the theoretical results of Fedorov (1972, Theory of Optimal Experiments, Academic Press) and MacKay (1992, Neural Computation, 4, 590-604), I apply techniques from optimal experiment design (OED) to guide the query/action selection of a neural network learner. I demonstrate that these techniques allow the learner to minimize its generalization error by exploring its domain efficiently and completely. I conclude that, while not a panacea, OED-based query/action selection has much to offer, especially in domains where its high computational costs can be tolerated. Copyright 1996 Elsevier Science Ltd