On the neural substrates for exploratory dynamics in basal ganglia: a model.

We present a neural network model of basal ganglia that departs from the classical Go/NoGo picture of the function of its key pathways-the direct pathway (DP) and the indirect pathway (IP). In classical descriptions of basal ganglia function, the DP is known as the Go pathway since it facilitates movement and the IP is called the NoGo pathway since it inhibits movement. Between these two regimes, in the present model, we posit that there is a third Explore regime, which denotes random exploration of the space of actions. The proposed model is instantiated in a simple action selection task. Striatal dopamine is assumed to switch between DP and IP activation. The IP is modeled as a loop of the subthalamic nucleus (STN) and the globus pallidus externa (GPe), capable of producing chaotic activity. Simulations reveal that, while the system displays Go and NoGo regimes for extreme values of dopamine, at intermediate values of dopamine, it exhibits a new Explore regime denoting a random exploration of the space of action alternatives. The exploratory dynamics originates from the chaotic activity of the STN-GPe loop. When applied to the standard card choice experiment used in the imaging studies of Daw, O'Doherty, Dayan, Seymour, and Dolan (2006), the model favorably describes the exploratory behavior of human subjects.