Serotonin regulates performance nonmonotonically in a spatial working memory network.

The prefrontal cortex (PFC) contains a dense network of serotonergic [serotonin, 5-hydroxytryptamine (5-HT)] axons, and endogenous 5-HT markedly modulates PFC neuronal function via several postsynaptic receptors. The therapeutic action of atypical antipsychotic drugs, acting mainly via 5-HT receptors, also suggests a role for serotonergic neurotransmission in cognitive functions. However, psychopharmacological studies have failed to find a consistent relationship between serotonergic transmission and cognitive functions of the PFC, including spatial working memory (SWM). Here, we built a computational network model to investigate 5-HT modulation of SWM in the PFC. We found that 5-HT modulates network's SWM performance nonmonotonically via 5-HT1A and 5-HT2A receptors, following an inverted U-shape. This relationship may contribute to blur the effects of serotonergic agents in previous SWM group-based behavioral studies. Our simulations also showed that errors occurring at low and high 5-HT concentrations are due to different network dynamics instabilities, suggesting that these 2 conditions can be distinguished experimentally based on their distinct dependency on experimental variables. We inferred specific predictions regarding the expected behavioral effects of serotonergic agents in 2 classic working-memory tasks. Our results underscore the relevance of identifying different error types in SWM tasks in order to reveal the association between neuromodulatory systems and SWM.