Boundary-limited serotonergic influences on pattern organization in rat sensory cortex.

In neonatal rodents, elevated levels of cortical serotonin (5-HT) blur the normally segmented vibrissae-related pattern of thalamocortical afferents (TCAs) in the posteromedial barrel subfield (PMBSF) of primary somatosensory cortex. We employed 5-HT immunocytochemistry or anterograde transport of 1'1'-dioctadecyl-3,3,3',3' tetramethyl-indocarbocyanin (Di-I) to label TCA arbors to study the effects of 5-HT manipulations on space occupied by TCAs within the PMBSF and the total area labeled. In rats treated to increase cortical 5-HT from birth to postnatal day (P) 6, the percentage of PMBSF area occupied by terminal labeling was significantly higher from that in controls (79.0% versus 23.7%, P < 0.05) for the highest levels of cortical 5-HT and was raised, although not significantly, for lower levels of 5-HT. The TCA coverage was significantly correlated with treatment dose. In animals exposed to a selective 5-HT1B agonist, 5-nonyloxytryptamine, or elevated endogenous 5-HT, the total areas of TCA aggregates in the PMBSF and those in visual cortex were similar to the controls. These results suggest that TCAs have a graded response to increasingly higher 5-HT concentrations. The lack of TCA expansion beyond normal cortical areas further implies that 5-HT-induced axon outgrowth is restricted at cortical boundaries.