A novel multichoice touchscreen paradigm for assessing cognitive flexibility in mice.

Cognitive flexibility refers to various processes which enable behaviors to be modified on the basis of a change in the contingencies between stimuli or responses and their associated outcomes. Reversal learning is a form of cognitive flexibility which measures the ability to adjust responding based on a switch in the stimulus-outcome contingencies of, typically two, perceptually distinct stimuli. Reversal tasks have provided valuable insight into the neural basis of cognitive flexibility, implicating brain regions including the lateral orbitofrontal cortex (lOFC) and dorsomedial prefrontal cortex (dmPFC). However, with two-stimulus reversal, it is difficult to determine whether response errors are due excessive perseveration, deficient learning, or other problems with updating. To address this limitation, we developed a mouse three-choice touchscreen-based visual reversal task, in which the contingencies of two stimuli were switched on reversal but a third, simultaneously presented, stimulus was never reinforced. We found that, in male C57BL/6J mice, responding at the previously rewarded stimulus predominated over the newly and never-reinforced stimuli during early reversal. Next, we showed that acute pharmacological inhibition of lOFC, but not dmPFC, impaired early reversal performance, relative to noninactivated controls. Interestingly, however, lOFC inactivation deficits were characterized by increased choice of the never-reinforced stimulus and a decrease in (perseverative-like) responding at the previously rewarded stimulus. These effects are inconsistent with the historical notion of lOFC mediating response inhibition and closer to recent views of the lOFC's role in response/outcome tracking. Overall, these findings provide initial support the utility of this novel paradigm for studying cognitive flexibility and its underlying neural substrates. Published by Cold Spring Harbor Laboratory Press.

Cognitive inflexibility in addictions including alcohol use disorders (AUDs) can prevent effective disengagement from destructive patterns of drug-seeking (Jentsch and Taylor 1999; Belin et al. 2016). In laboratory settings, measures of cognitive flexibility such as reversal learning are slowed in AUD patients (Vanes et al. 2014; Le Berre et al. 2017), while rodents chronically exposed to alcohol or other drugs of abuse (e.g., cocaine), exhibit abnormalities in reversal performance and other forms of cognitive flexibility, including attentional set-shifting (Schoenbaum et al. 2004; Coleman et al. 2012; DePoy et al. 2013; Trantham-Davidson et al. 2014; Hu et al. 2015; Varodayan et al. 2018). In turn, these behavioral disturbances have been attributed to drug-induced adaptations in certain cortical and striatal regions that are known substrates for these cognitive processes (Schoenbaum and Shaham 2008; Moorman 2018).

Prior studies in the rodent and nonhuman primate have shown that neurons in the lateral orbitofrontal cortex (lOFC) exhibit correlates of successful reversal learning, while experimentally induced disruptions of lOFC function impair reversal (Iversen and Mishkin 1970; Jones and Mishkin 1972; Dias et al. 1996, 1997; Rolls 1996; Tremblay et al. 1998; Schoenbaum et al. 2002, 2003; Chudasama and Robbins 2003; Izquierdo et al. 2004; Walton et al. 2004, 2010; Kim and Ragozzino 2005; Stalnaker et al. 2006; Boulougouris et al. 2007; Clarke et al. 2008; Ghods-Sharifi et al. 2008; Brigman et al. 2013; Bissonette et al. 2014; Dalton et al. 2016; Marquardt et al. 2017). However, although lOFC disruptions reliably disrupt reversal, differing views regarding what this reflects about the function of the region exist. Early hypotheses suggested the lOFC was important for response inhibition (Jones and Mishkin 1972), while more recent accounts posit a role in tracking response and outcome histories to guide decisions, and in using stimuli-associated specific outcomes to make choices among the available options (Balleine et al. 2011; Noonan et al. 2012; Rudebeck and Murray 2014; Costa et al. 2015; Stalnaker et al. 2015; Izquierdo et al. 2016).

In addition to the prominent role ascribed to the lOFC, the medial prefrontal cortex (mPFC), as well as a number of other brain regions such as the dorsal striatum, have been shown to subserve reversal (Palencia and Ragozzino 2004, 2006; Ragozzino and Choi 2004; Tzavos et al. 2004; Brown et al. 2010; Graybeal et al. 2011; Amodeo et al. 2017; Grospe et al. 2018). For example, lOFC lesions in rats produced a perseverative-like reversal deficit (increased early reversal responding at the previously rewarded stimulus [Sprior]), whereas mPFC lesions appeared to impair learning of the new stimulus–reward contingency (more late-reversal responding at the Sprior) (Chudasama and Robbins 2003). However, other studies found that while mPFC lesions or inactivations disrupted attentional shifts, there were null, and even facilitatory, effects on reversal (Dias et al. 1996; Birrell and Brown 2000; McAlonan and Brown 2003; Ragozzino et al. 2003; Brigman and Rothblat 2007; Bissonette et al. 2008, 2013; Floresco et al. 2008; Dalton et al. 2016).

Some of the inconsistencies and interpretative issues evident in the literature might stem in part from limitations inherent to the most typically used reversal paradigms. In such preparations, subjects learn the outcome contingencies of two stimuli, which are then switched on reversal. Because both stimuli have well-established antecedent reward versus nonreward associations at reversal, it is difficult to disambiguate perseverative responding from impaired learning about the new contingencies or other problems, such as deficient on-task response/outcome tracking. This method also lacks the possibility to evaluate the presence of irrelevant/distracting stimuli that are known to potently influence human decision-making in a manner related to OFC and mPFC recruitment (Chau et al. 2014).

One approach to circumventing some of these problems is to expand the range of choices available on reversal by adding a third (or more) stimulus that is never rewarded (Bussey et al. 1997; Ragozzino et al. 2003; Kim and Ragozzino 2005; Lee et al. 2007; Ragozzino and Rozman 2007; Rudebeck et al. 2008; Seu et al. 2009; Walton et al. 2010; D'Cruz et al. 2011; Kosaki and Watanabe 2012; Noonan et al. 2012, 2017; Riceberg and Shapiro 2012). Yet despite rapid advances in the availability of powerful genetic tools to monitor and manipulate neural substrates of higher-order behaviors in mice, there is currently a lack of multistimulus reversal paradigms to test cognitive flexibility in this species.

The goal of the present study was to first develop a novel, three-choice, version of a previously described mouse visual discrimination and reversal touchscreen paradigm (Izquierdo et al. 2006; Mar et al. 2013), and then to evaluate the effects on early reversal performance after inactivating the lOFC and dorsomedial prefrontal cortex (dmPFC). We began by assessing a cohort of male C57BL/6J mice (a commonly used inbred strain) using a modified version of the Bussey–Saksida Touch Screen System (Graybeal et al. 2014) with three response windows on a touchscreen panel, within each of which was presented a single two-dimensional visual stimulus (Fig. 1A; see Supplemental Material for full experimental procedures).

Figure 1.

Performance in a novel three-choice visual discrimination and reversal touchscreen paradigm. (A) A mouse-eye (upper) and overhead (lower) view of the apparatus. (B) Visual stimuli used for discrimination and reversal (note, the designation of bars and dots as the S+ on discrimination, and the corresponding Sprior designation on reversal, was counterbalanced across mice and did not affect performance). (C) Behavioral raster from single training sessions of a representative mouse at each relevant training phase. Blue lines represent choice of the S+ or Sprior during discrimination (left) or reversal (right) sessions. Red lines represent choice of the S1− (discrimination sessions, left) or Snew (reversal sessions, right), while green lines represent the S2− (discrimination sessions, left) or Snever (reversal sessions, right). The maximum session length was 3600 sec (1 h). Note the shift from random stimulus selection, to efficient discrimination displayed across discrimination, and then the marked degree of perseveration in early reversal, followed by characteristic sampling and then reversal during the middle and late reversal phases, respectively. (D) The percentage of S+/Sprior responses increased across discrimination and remained high at early reversal before decreasing across subsequent reversal training (task × phase interaction: F(2,26) = 217.9, P < 0.0001; effect of task: F(1,13) = 0.68, P = 0.4245; effect of phase: F(2,26) = 9.51, P = 0.0008, followed by individual comparisons via Tukey's post-hoc tests). (E) The percentage of S1−/Snew responses decreased from early to late discrimination and progressively increased over reversal (task × phase interaction: F(2,26) = 239.1, P < 0.0001; effect of task: F(1,13) = 34.32, P < 0.0001; effect of phase: F(2,26) = 24.25, P < 0.0001, followed by Tukey's post-hoc tests). (F) The percentage of S2−/Snever choice was low by late discrimination and did not change during reversal (task × phase interaction: F(2,26) = 4.01, P = 0.0302; effect of task: F(1,13) = 43.44, P < 0.0001; effect of phase: F(2,26) = 22.52, P < 0.0001, followed by Tukey's post-hoc tests). (G) Strings of consecutive S+/Sprior responses increased over discrimination and decreased over reversal (task × phase interaction: F(2,26) = 5.50, P < 0.0102; effect of task: F(1,13) = 26.26, P = 0.0002; effect of phase: F(2,26) = 19.86, P < 0.0001, followed by Tukey's post-hoc tests). (H) Strings of S1−/Snew responses were decreased by late discrimination before increasing across reversal (task × phase interaction: F(2,26) = 91.38, P < 0.0001; effect of task: F(1,13) = 34.57, P < 0.0001; effect of phase: F(2,26) = 25.87, P < 0.0001, followed by Tukey's post-hoc tests). (I) S2−/Snever strings were reduced at late discrimination and stayed flat across reversal (task × phase interaction: F(2,26) = 2.39, P = 0.1116; effect of task: F(1,13) = 14.92, P = 0.0020; effect of phase: F(2,26) = 10.03, P < 0.0006, followed by Tukey's post-hoc tests). n = 14. Data are means ± SEM. (*) P < 0.05.

To characterize the task, mice were first pretrained to reliably make a single touch at one response window to obtain a food-pellet reward (∼10 sessions). During discrimination sessions, three novel stimuli were simultaneously presented on each trial, with responses at one stimulus (S+, “bars” or “dots” counterbalanced across mice) producing reward at a continuous rate of reinforcement, followed by another trial in which the stimuli were presented in a randomly selected spatial configuration (Fig. 1B). Responses at either of the other two stimuli (S1−, S2−) produced no reward and a 15-sec timeout period (signaled by extinguishing the house lights), followed by a correction trial in which the stimuli were presented in the same spatial configuration. Correction trials were repeated until a correct response was made. Testing proceeded on daily sessions comprising either 36 (for task characterization) or 30 (for inactivation) trials (excluding correction trials) until a criterion of >75% correct performance on two consecutive sessions was attained (=11.9 ± 1.4 sessions).

Reversal began on the next session after discrimination criterion was achieved. Here, the same three stimuli were presented, but the previously rewarded stimulus was now unrewarded (S+ now Sprior), while one of the previously unrewarded stimuli (“moon” for all mice) was now rewarded (S1− now Snew) and the other stimulus remained unrewarded (S2− now Snever) (Fig. 1B). The procedure was otherwise the same as for discrimination (including use of correction trials). Testing proceeded until a >75% correct performance on two consecutive sessions criterion was met (=11.2 ± 1.2 sessions). For each mouse, sessions to discrimination criterion and, separately, reversal criterion were tallied and subdivided into early, mid, and late phases (Bergstrom et al. 2018). Of the total responses made during each phase, the percentage of responses at each of the three stimuli was calculated. In addition, to evaluate the microstructure of responding at each phase, the average length of unbroken strings of responding at the same stimulus was also analyzed for each phase (Brigman et al. 2013).

Results showed that choice of the S+/Sprior stimulus increased significantly from early to late discrimination and remained high at early reversal before decreasing across subsequent reversal training (Fig. 1D; e.g., behavioral raster plots, see Fig. 1C). Conversely, selection of the S1−/Snew decreased from early to late discrimination and progressively increased over reversal, while S2−/Snever choice was low by late discrimination and spiked during mid-reversal—apparently reflecting sampling of both previously unrewarded options at this mid-stage (Fig. 1E,F). The patterns evident in overall percent responding were substantiated by the trial-by-trial analysis showing that strings of consecutive S+/Sprior responses increased over discrimination and decreased over reversal. Strings of S1−/Snew responses, in contrast, were low by late discrimination before increasing across reversal, whereas S2−/Snever strings were also reduced by late discrimination and remained flat across reversal (Fig. 1G–I). The number of Sprior (r = +0.72, P < 0.003) and Snever (r = +0.55, P < 0.044), but not Snew (r = +0.39), responses made at early reversal positively correlated with Snew choice at late reversal—suggesting that greater negative feedback early in the task led to superior performance later (Supplemental Fig. S1A–C). Lastly, choice latencies for all three stimuli quickened across the phases of discrimination and again at reversal (Supplemental Fig. S2).

These initial data establish the feasibility of assessing multichoice reversal in mice using a touchscreen-based procedure. They also demonstrate this paradigm's value in dissociating patterns of errors that are perseverative in nature (Sprior > Snew) from those reflecting random responding or exploration of the alternative stimulus options (Sprior = Snew). On the basis of these findings, we next evaluated the consequences of pharmacologically inactivating either the lOFC or dmPFC during early reversal, given the known contributions of these regions to cognitive flexibility (Fig. 2A). Following pretraining, male C57BL/6J mice were implanted with indwelling guide cannula into the lOFC (Fig. 2B) or dmPFC (Fig. 2E) and trained to the discrimination criterion in 12–15 sessions. To inhibit neuronal activity during early reversal, the GABA receptor agonist, muscimol (MUS), or saline (SAL) vehicle was bilaterally infused into the lOFC or dmPFC prior to each of the first two reversal sessions (capturing early reversal).

Figure 2.

Inactivation of the lOFC, but not dmPFC, disrupts early three-choice reversal performance. (A) Experimental timeline for pretraining, discrimination testing, and early reversal inactivation. (B) Ventral extent of the infusion located in lOFC. (C) Inactivation of lOFC resulted in a decrease in the percentage of trials ending in Sprior choices (left panel; corrected t(16) = 3.09, P < 0.0210), without affecting the percentage of Snew trials (middle panel; corrected t(16) = 0.80, P > 0.05). In contrast, the percentage of trials ending in Snever choices were increased following lOFC inactivation (right panel; corrected t(16) = 2.56, P < 0.0412). (D) lOFC inactivation decreased the average length of strings of Sprior (left panel; t(16) = 2.72, P = 0.0151) without affecting strings of Snew responses (middle panel; t(16) = 0.24, P = 0.8157). The length of strings of Snever responses tended to be increased by lOFC inactivation (right panel; t(16) = 2.12, P = 0.0502). (E) Ventral extent of the infusion located in dmPFC. (F,G) Inactivation of the dmPFC did not significantly affect any measure (all t-values < 1.5, all P-values > 0.15). n = 9–10 per group. Data are means ± SEM. (*) P < 0.05.

lOFC inactivation produced significant alterations in the pattern of responding during early reversal, as well as a reduction in the number of trials performed overall, relative to saline-infused controls (SAL = 72.2 ± 4.9, MUS = 33.4 ± 8.2; t(16) = 4.03, P < 0.0010). Specifically, mice with the lOFC inactivated made significantly fewer responses at the Sprior than controls but showed a significant increase in choices at the Snever, as a percentage of total trials completed. Selection of the Snew option was equivalently low across the inactivated and saline groups (Fig. 2C). Microstructural analysis of trial-by-trial responding revealed that lOFC-inactivated mice made significantly shorter strings of consecutive of Sprior responses, with a corresponding increase in Snever strings (and no change in Snew strings) (Fig. 2D). Lastly, the latency to choose the Sprior, but not Snew or Snever stimulus, was longer after lOFC inactivation (Supplemental Fig. S2). The fact that choice latency was selectively higher for the Sprior stimulus after lOFC inactivation argues against the kind of general loss of response vigor that has been reported in rats studies (St Onge and Floresco 2010; Dalton et al. 2016). It could instead reflect deliberation or hesitancy at the Sprior option. Why this would be greater after lOFC inactivation is unclear; one intriguing possibility is that this renders animals more sensitive to the Sprior’s strong violation of outcome expectancies.

These data demonstrate that early reversal performance in this three-choice task was disrupted by lOFC inactivation. However, the pattern of effects does not fit with the historically favored notion that the lOFC mediates response inhibition—which would be expected to produce an increase, not a decrease, in perseverative-like responding at the Sprior. Rather, these data are more readily explained by more recent views that the lOFC integrates a record of choices and their respective outcomes to guide the optimal response based on the calculated value of relative options available (Noonan et al. 2012; Rudebeck and Murray 2014; Stalnaker et al. 2015; Izquierdo et al. 2016). For example, multichoice reversal studies in nonhuman primates find that, without the normal contribution of the lOFC (or mPFC), animals do not perseverate more, but instead more frequently shift between choices (Walton et al. 2010; Noonan et al. 2012, 2017; Chau et al. 2014).

This effect has been explained by an inability to integrate recent and historical choice-outcomes—which are mixed and conflicting during reversal—to support decisions, leading to the misattribution of positive outcomes to the unrewarded stimuli (“deficient credit assignment”). While a deficiency of this kind could conceivably contribute to the reversal abnormalities observed in the current study, it cannot adequately account for the specific pattern we see. In particular, if mice had difficulty with credit assignment per se, then we would expect to see an increase in responding at the Snew, as well as the Snever, because credit misattributed to the latter would derive from more frequent choice of the former. To decipher the nature of the effects we do see, it could be informative to inactivate the lOFC at the mid-reversal stage, when Snew responding is higher than at early reversal, and ask whether Snever responding is still excessively high after inactivation.

The current data also bear comparison with prior rodent studies of lOFC inactivation in reversal tasks that vary in stimulus number (two or more), stimulus type (visual, olfactory, or spatial) and/or reinforcement schedule (deterministic or probabilistic). In rats, lOFC inactivation has been found to impair performance of a probabilistic spatial reversal, without affecting performance on a deterministic version (Dalton et al. 2016). Conversely, rat or mouse lOFC inactivation or NMDA receptor antagonism was reported to increase putative perseverative-like errors in two-choice odor (Kim and Ragozzino 2005) and visual touchscreen (Chudasama and Robbins 2003; Brigman et al. 2013) reversals, but cause a general increase in perseverative, regressive and irrelevant errors in a four-choice odor reversal (Kim and Ragozzino 2005; Ragozzino 2007). One preliminary conclusion to draw from these studies is that the manner in which lOFC disruptions behaviorally manifest critically depends on the range of options available and their relative reinforcement histories.

There are also some noteworthy differences between the current visual reversal task and prior studies that use olfactory, tactile and visual stimuli, often in compound. In the touchscreen procedure, mice perform many more trials to reach discrimination and reversal criteria than rats typically do in, for example, digging tasks (i.e., hundreds versus tens). As such, all the stimuli are highly familiar by reversal in our task, whereas rats are sometimes introduced to novel stimuli across serial discriminations and reversals. These differing levels of stimulus-familiarity could have important consequences for how reversals are performed and recruit the lOFC. Indeed, Tait and Brown (2007) found that juxtaposing two reversal stimuli based on their familiarity and relative reinforcement histories markedly affected the nature of lOFC loss-of-function effects in a rat digging task. lOFC-lesioned rats displayed poor learning and increased omissions (“refusals to dig”) when required to select the previously unrewarded stimulus over a novel option, but actually had superior learning when rewarded for choosing a novel stimulus over the previously reward option (Tait and Brown 2007). One interesting avenue for future studies will be dissecting the influence of these parameters. As a behavioral platform, the touchscreen provides a tractable means to do so, given that the number, familiarity, and reinforcement histories of the choice-options can be varied, and their influence examined across single as well as potentially serial reversals.

In contrast to the marked effects of lOFC inactivation, inactivating the dmPFC was without any discernible effect (Fig. 2E–G). Prior reports of dmPFC inactivation effects on reversal tasks (including multichoice) in rats have been mixed, with examples of improvement, impairment and, as in the current case, no change in performance (cf. Becker et al. 1981; Bussey et al. 1997; Ragozzino et al. 2003; Boulougouris et al. 2007; Ragozzino and Rozman 2007; Floresco et al. 2008; Dalton et al. 2016). Again, this might reflect important variants in task parameters. Tasks that require cross-modal cognitive flexibility and resolution of rule-conflict appear to be particularly sensitive to mPFC disruption (Ragozzino et al. 2003; Floresco et al. 2008; Bissonette and Roesch 2017). For example, rats with dmPFC lesions exhibited deficits in a Y-maze reversal after previously learning a strategy shift; an effect attributed to a deficit in maintaining performance under conditions of conflicting strategies (Oualian and Gisquet-Verrier 2010).

Following early reversal inactivation in the current study, mice were trained to reversal criterion (without further infusions). Groups took an equivalent number of sessions to reach reversal criterion (lOFC: SAL = 15.0 ± 2.8, MUS = 13.7 ± 1.1; mPFC: SAL = 10.3 ± 0.9, MUS = 10.3 ± 1.4), indicating that the initial manipulation left no residual effects on performance when the cortical regions were back “online.” The lOFC or dmPFC was then inactivated during a single post-criterion session to test for effects on the expression of the now well-learned stimulus–reward contingencies (Fig. 3A), but neither inactivation significantly affected overall percent stimulus-choice (Fig. 3B,D). lOFC, but not dmPFC, inactivation did produce a modest increase in the length of Sprior strings, while decreasing consecutive selections of the Snew option (Fig. 3C,E) and increasing the number of trials performed overall (SAL = 35.0 ± 1.2, lOFC = 45.1 ± 4.2; t(16) = 2.27, P < 0.0378). Also, and in contrast to the selective increase in Snew choice latency after early reversal lOFC inactivation, inactivation at late reversal increased latencies generally (Supplemental Fig. S3).

Figure 3.

Effects of lOFC and dmPFC inactivation on late three-choice reversal performance. (A) Experimental timeline for post-reversal criterion inactivation. (B) lOFC inactivation did not significantly affect the percentage of Sprior (left panel; corrected t(16) = 2.03, P = 0.1160), Snew (middle panel; corrected t(16) = 2.39, P = 0.0856), or Snever (right panel; corrected t(16) = 1.99, P = 0.1160) choices. (C) The average length of Sprior strings was increased by lOFC inactivation (left panel; t(16) = 2.61, P = 0.0190), while the average Snew string length was diminished by lOFC inactivation (middle panel; t(16) = 2.45, P = 0.0261). In contrast, Snever string length was not altered by lOFC inactivation (right panel; t(14) = 0.98, P = 0.3450). (D,E) Inactivation of the dmPFC at late reversal did not significantly affect any measure (all t-values <1.5, all P-values > 0.15). n = 9–10 per group. Data are means ± SEM. (*) P < 0.05.

The slight shift in favor of the Sprior after late reversal lOFC inactivation could reflect partial reversion to the discrimination associations (Delamater 2007) which, as noted above, could be expressed by other brain regions. However, the fact that correct choice was reduced and both error types were increased (not just the Sprior) suggests a problem with maintaining the stimulus–reward/nonreward associations formed during reversal. This is somewhat surprising considering that lOFC is typically implicated when behaviors are being adjusted (as in early reversal), rather than after adjustments have been made (as in late reversal) (Boulougouris et al. 2007). However, the lOFC was also found to support established reversal performance on a probabilistically reinforced reversal task conducted in rats (Dalton et al. 2016). It may be that the multichoice and probabilistic reversals are especially taxing on the lOFC's capacity for predicting and tracking stimulus–outcome associations, such that the region retains a hand in maintaining performance even after behavior has successfully adjusted to the new stimulus–reward contingencies. To address this question going forward, it would be valuable to chronically measure (e.g., via neuronal recordings or imaging) the sustained engagement of the lOFC across reversal learning in the three-choice task.

In conclusion, the current three-choice touchscreen task represents novel paradigm for assessing murine reversal learning and cognitive flexibility that may be useful for translational research given its overt similarity to cognitive testing procedures used in higher species, including humans (Mar et al. 2013; Akaishi et al. 2016; Izquierdo et al. 2016; Noonan et al. 2017). Further speaking to this potential, we show that the reversal task is highly sensitive to disruption of the lOFC, a brain region subserving the modification of choices to accommodate new outcome contingencies, and implicated in narrowed, intransigent patterns of behavior characteristic of AUDs and other addictions.