3D Printable Device for Automated Operant Conditioning in the Mouse.

Operant conditioning (OC) is a classical paradigm and a standard technique used in experimental psychology in which animals learn to perform an action to achieve a reward. By using this paradigm, it is possible to extract learning curves and measure accurately reaction times (RTs). Both these measurements are proxy of cognitive capabilities and can be used to evaluate the effectiveness of therapeutic interventions in mouse models of disease. Here, we describe a fully 3D printable device that is able to perform OC on freely moving mice, while performing real-time tracking of the animal position. We successfully trained six mice, showing stereotyped learning curves that are highly reproducible across mice and reaching >70% of accuracy after 2 d of conditioning. Different products for OC are commercially available, though most of them do not provide customizable features and are relatively expensive. This data demonstrate that this system is a valuable alternative to available state-of-the-art commercial devices, representing a good balance between performance, cost, and versatility in its use.

Significance Statement

3D printing is a revolutionary technology that combines cost-effectiveness with an optimal trade-off between standardization and customization. Here, we show a device that performs operant conditioning (OC) in mice using largely 3D-printed parts. This tool can be employed to test learning and memory in models of disease. We expect that the open design of the chamber will be useful for scientific teaching and research as well as for further improvements from the open hardware community.

Introduction

Operant conditioning (OC; Jones et al., 1939) is a standard technique used in experimental psychology in which animals, like rodents (Francis and Kanold, 2017; O'Leary et al., 2018), reptiles (Mueller-Paul et al., 2014), birds (Cook, 1992), dogs (Range et al., 2008), monkeys (Range et al., 2008), and humans (Siqueland, 1964; Angulo-Barroso et al., 2017), learn to perform an action to achieve a reward. By using this paradigm, it is possible to extract learning curves and measure accurately mental chronometry [e.g., reaction times (RTs)]. As previously suggested (Escobar and Pérez-Herrera, 2015; Francis and Kanold, 2017; O'Leary et al., 2018), different products for OC are commercially available, though most of them do not provide customizable features and are relatively expensive. Neuroscience research has greatly benefited from new 3D printing technologies bringing new possibilities to build tools, and increasing productivity and user timeliness. 3D printing also opened unprecedented resources for training students and solving common experimental problems (Baden et al., 2015). There is a plethora of work using 3D-printed mechanical parts (Baden et al., 2015), ranging from fluorescence microscopes (Chagas et al., 2017) to electrophysiology systems (Siegle et al., 2017). The combination of 3D printing with off-the-shelf, low-cost optical and electronic components facilitates reproducibility of experimental tools internationally and promotes rapid iteration and prototyping (Chagas et al., 2017). Here, we demonstrate an affordable, fully 3D printable, and automated solution that can be reproduced rigorously in any laboratory equipped with a 3D printer with a total cost around 160 € (Table 1). We designed the chamber entirely using 3D modeling for several reasons: first, it has a high degree of reproducibility, since the model is standardized and can be downloaded to print the same structure with the same materials throughout different laboratories. Second, it can be easily customized in relation with specific experimental needs. Lastly, it can be easily shared through on-line repositories. With these cost-efficient and accessible components, we assayed the possibility to perform two-alternative forced choice OC using audiovisual cues while tracking in real time mouse position.

Table 1

Bill of materials

Material	Price (€)	Vendor	Code	Manufacturer
LED MATRIX	26.74	amazon.it	B071VJL91V	Kuman:WS01
Stepper motor	3.38	amazon.it	B00DGNO6PI	Elegoo
PLA	16.66	amazon.it	B06W568X1G	TECHNOLOGY OUTLET
Pi camera	18.99	amazon.it	B07P8PG5MF	Bewinner: Bewinnertyv48w6mf5
Raspberry PI	44.51	amazon.it	B01CD5VC92	raspberrypi
Graphene PLA	27.50	filoprint.it	PLA_GRAFENE_175	Haydale
Arduino UNO	16.85	amazon.it	B07SL2W4CL	Arduino: A000066
Power supply	5.69	amazon.it	B00UVOHJ0Y	Samsung:TA10EWE
Piezo Buzzer	1.35	adafruit.com	PS1240	tdk
Cables/wires	2.00	Off the shelf
Total	163.67

Materials and Methods

Mice housing and handling

Animals were kept at a constant temperature (22°C) with a standard 12/12 h light/dark cycle (7 A.M. to 7 P.M.). Food was available ad libitum and changed weekly. During OC protocol, mice are water restricted (body weight >85%; Goltstein et al., 2018) of their baseline. Before the experiment mice are handled for 1 h/d for one week. After the last daily session, mice had free access to water for 1 h (23 h of water deprivation). All the experiments were conducted in accordance with the directives of European Community Council (2011/63/EU) and approved by the Italian Ministry of Health. We tested six wild-type C57BL/6J [from postnatal day (P)50 to P180, four female and two male mice; Charles River].

3D-printed OC chamber

The OC arena (16 × 16 × 16 cm; thickness, 3 mm; Fig. 1) is 3D printed using gray or white PLA (B06W568X1G, Technology Outlet). The 3D project is designed using FreeCAD software, exported in stereolithography (STL) format, converted to G-code using Cura (https://ultimaker.com/software/ultimaker-cura) and printed using Kentstrapper Verve 3D printer (https://kentstrapper.com/stampante-3d-kentstrapper-verve/). In Figure 1, an exploded-view drawing of the chamber is shown. The color coding corresponds to different components of the apparatus (visual stimulation parts in red; camera holder in green; syringe pump in purple). All these components are coated using epoxy transparent resin (LF-L2GR-26GX, resinpro), which allows cleaning (5% ethanol in water). The arena front wall contains the elements interfacing the animal with the computer. It can be modularly assembled to the arena and is composed by a squared frame containing the LED matrix at the center, four holes for the touch buttons in the lower part, a central hole for the lick spout and a hole in the upper part to connect a piezo buzzer. The touch buttons are printed using graphene PLA (PLA_GRAFENE_175, filoprint), and connected using conductive glue (Chemtronics: CW2400) to a female pin (B07XQHD752, amazon.it) using a resistor (25 MΩ). A dotted grid is interposed between the LED matrix and the inside of the chamber and has two roles: first, the dotted pattern restricts the visibility of the LED lights to equal small circles; second, it contains a grid of walls facing the LEDs that prevents the light from each source to spill over to the neighboring dots. The LED matrix is covered with a thin white Plexiglas foil, so that single LED are not visible if they are off and to diffuse light uniformly. The camera holder, is joint assembled on top of the frontal wall and it is designed to maintain the camera at the distance necessary to image the entire arena using a 3.6-mm focal length objective. The syringe pump is composed of a base that fixes the barrel of the syringe into position and of a piston that slides on a stepper motor guided M8 metal screw and allows to push or pull the plunger.

Figure 1.

3D printable OC apparatus. , left, Top-view of the apparatus. Center, Interface wall. Right, Blue dotted stimuli, camera holder, and syringe pump. , An animal during the task, the blue line delimitates the “active zone.” , An exploded view of the project showing the assembling scheme. , Circuit diagram of all the components.

Hardware

An electronic board is mounted on a grounded metal sheet and is composed by a Raspberry Pi connected via USB to an Arduino UNO (AU) board (https://store.arduino.cc/arduino-uno-rev3). The Raspberry Pi (https://www.raspberrypi.org/products/raspberry-pi-3-model-b/) acts as the main computer of the setup. It executes the Python three script that handles the structure of the experiment, performs computer vision using a Raspberry PI camera (Bewinner: Bewinnertyv48w6mf5), and saves data (Fig. 1). The AU controls sensors and actuators in the OC chamber. Two touch buttons, made using conductive PLA, acts as capacitive sensors and are connected to AU using coaxial cables (3 mm in diameter) to minimize environmental noise. The main advantage of using graphene PLA resides in the possibility to print different button designs (e.g., for motor impairment, nose poking, etc.). There are three actuators: a LED matrix serves as display (Adafruit: 1487), a piezo buzzer (Adafruit: PS1240, frequency range: 2–10 kHz, 60 dB) is used as acoustic stimulator glued at the top of the frontal door, and a stepper motor (amazon.it: 28BYJ-48, with ULN2003) connected to a M8 screw guiding the piston of a syringe pump controlling a disposable syringe (10 ml) connected with a silicone tube equipped by luer tapers adapters to a blunt needle (Warner Instruments: SN-18) for reward delivery. This modular configuration allows the proper cleaning of the delivery tubing after each session. We use an external 5-V 2-A DC power supply (Samsung: TA10EWE) with a 1000-μF capacitor to power the LED matrix and the stepper motor. A diagram of the electrical wiring is shown in the Figure 1.

Software

AU program

The code controlling the OC box is organized in four files, the file called skinner.ino contains the logic of the experiment and manages the serial communication with the computer. Different files are dedicated to different aspects of the program: the file called button.ino contains functions to control the touch buttons and play auditory stimuli, the file called ledLib.ino contains wrapper functions to control Adafruit NeoPixel library (https://www.adafruit.com/product/1487) and generate simple visual stimuli easily, the third file called stepper.ino, contains functions to control the syringe pump using the Arduino Stepper motor library (https://www.arduino.cc/en/reference/stepper). In summary, to setup the AU, a user needs to download the folder containing the .ino files, uncompress and upload the file skinner.ino.

Raspberry Pi program

On the Raspberry Pi, a Python script controlling the experiment has been written using IDLE. The program relies on a number of external libraries that are required to run all parts of the script with no errors. Since the task relies on real time tracking of the animal position we use picamera and opencv libraries to acquire frames and process them using K-nearest neighbors based background-foreground segmentation (Zivkovic and van der Heijden, 2006), a widely used algorithm for generating a foreground mask using static cameras (Fig. 2). The technique consists of two main steps, the first one is the background initialization in which we use 1000 frames of the empty arena, then we set the learning rate to zero and the algorithm stops updating the background so it is ready to locate reliably the position of the animal with a frame rate of 20 Hz. LibSerial library is used to communicate with the AU during the task sending symbolic codes and changing the state of the AU in the OC chamber. We used Tkinter library to write the initial GUI to set the experimental parameters. The behavioral sequence is outlined in Figure 2. Virtually the chamber can be divided into two sections: the anterior part that contains the interface between the mouse and the computer, and the posterior side that is designed as an active area to activate the trials. If the mouse remains in the active area for a given amount of time (1.5 s) the trial is triggered. At this stage a visual stimulus is shown on the display and the system waits for animal response. When the mouse touches one of the two buttons, an auditory feedback is produced, with a tone that varies depending on whether the answer is correct (3300 Hz) or wrong (2700 Hz). In case of correct answer, a drop (7 μl) of water with 1% condensed milk is released.

Figure 2.

Behavioral procedures. , The detection of the mouse is obtained using background subtraction from the current frame and then applying a threshold, isolating only the mouse silhouette. , Behavioral sequence to obtain the reward. , Diagram showing the behavioral procedures, during the shaping phase and the operating task. , Flowchart of the assisted procedure.

Implementation of an LCD screen

As a proof of principle of customizability, we added a version of the OC chamber that is able to show more complex visual stimuli. This version includes an edit of the frontal wall that can host a TFT monitor (Kookye 3.5” for RPI3) and a folder (LCD_oc_chamber) containing code that runs on Psychopy2 (Peirce, 2008), a Python package dedicated to behavioral experiments. This configuration allows to show RGB images as visual stimuli.

Code accessibility

The code described in the paper is freely available online at https://github.com/raffaelemazziotti/oc_chamber. The code is also available as Extended Data 1.

zip contains the code for both Arduino and Raspberry Pi boards. Download Extended Data 1, ZIP file.

The OC protocol

Familiarization

This phase is conducted by placing each animal in the OC box for three sessions of 10 min, spaced by at least 2 h between each other. During this phase, a liquid reward, coupled with the “correct” tone is provided manually whenever the mouse is in the active area, in this way the animal learns where to find the reward and associate it with the tone.

Shaping (3 d)

The visual stimulus is introduced (Fig. 2). It consists of two bright (0.9 cd/m2) blue (465–475 nm) dots (5 mm) that appear above the two buttons. The mouse needs to touch one of the two buttons to obtain the reward.

Operant task (OT, 5 d)

During this phase, only one dot appears, identifying the correct button. If the mouse touches the correct button, the correct tone is reproduced and the animal receives the reward. If the mouse touches the wrong button, the “wrong” tone is reproduced and no liquid reward will be delivered. This procedure is shown in Figure 2. The sequence of stimuli is balanced so that the mouse sees each case the same number of times. In order to prevent perseveration with the same answer, during the first 2 d we adopted an assisted procedure (Fig. 2): the first stimulus presented is random, if the mouse produces the correct answer the following stimulus is randomized, in case of wrong answer instead, the system repeats the same stimulus until the mouse gives the correct answer three times.

Follow-ups

In order to test the ability of the mouse to recall the task, we tested animals in different follow-ups, respectively, at 6 d, 27 d, three months, and four months approximately after the end of OT. For each recall sessions we tested mice once per day.

Data analysis and statistics

Data processing is performed using Python and the statistical analysis (Table 2) using GraphPad Prism 7. To analyze mouse tracking data the arena is virtually divided into 256 (16 × 16) bins and raw exploration is z-scored to obtain relative exploration measures. To quantitatively test whether the mouse preferentially explores some of the bins, we constructed a resampled binned exploration matrix representing chance level for each bin, randomly permuting each animal exploration matrix for 100 times. The software Fritzing was used to draw the wiring diagram of the electrical components. We used Rhino 6 to draw the exploded version of the model.

Table 2

Statistical table

Figure	Type of test	Statistical data
Fig. 3A, average trials	RM one-way ANOVA, Dunnett’s multiple comparisons post hoc	p = 0.0006, post hoc day 1 vs day 3, p < 0.001
Fig. 3A, RT	As above	p = 0.0002, day 1 vs day 2, p = 0.001 and day 1 vs day 3, p = 0.0002
Fig. 3A, ITI	As above	p = 0.0002, day 1 vs day 2, p = 0.0148 and day 1 vs day 3, p = 0.0001
Fig. 3C, average trials	As above	p = 0.0046, day 1 vs day 3, p = 0.0057; day 1 vs day 4, p = 0.0057 and day 1 vs day 5, p = 0.0036
Fig. 3C, RT	As above	p = 0.0022, day 1 vs day 3, p = 0.0027; day 1 vs day 4, p = 0.0011 and day 1 vs day 5, p = 0.0275
Fig. 3C, ITI	As above	p < 0.0001, day 1 vs day 3, p = 0.0004; day 1 vs day 4, p < 0.0001 and day 1 vs day 5, p = 0.0002
Fig. 3C, correct	As above	p = 0.0025, day 1 vs day 2, p = 0.0464; day 1 vs day 3, p = 0.0043; day 1 vs day 4, p = 0.0042 and day 1 vs day 5, p = 0.0013
Fig. 3D, average trials	As above	p = 0.0058; baseline vs 4 months, p = 0.0042
Fig. 3D, RT	As above	p < 0.0001, baseline vs 6 d, p = 0.0063; baseline vs 4 months, p < 0.0001
Fig. 3D, ITI	As above	p < 0.0001, baseline vs 27 d, p = 0.0093, baseline vs 3 months, p = 0.0009, baseline vs 4 months, p = 0.0252
Fig. 3D, correct	As above	p = 0.2290
Fig. 4B, relative exploration	As above	p < 0.0001, corner vs center, p < 0.0001; corner vs reward, p < 0.0001, center vs reward, p < 0.0001
Fig. 4C, correlation matrix TR vs RT	Spearman’s correlation	r = –0.8223 (95% CI, –0.9335 to –0.5669); R2 = 0.6761
Fig. 4C, correlation matrix TR vs ITI	Spearman’s correlation	r = –0.9472 (95% CI, –0.9811 to –0.8573); R2 = 0.8971
Fig. 4C, correlation matrix ITI vs RT	Spearman’s correlation	r = 0.9209 (95% CI, 0.7910 to 0.9714); R2 = 0.8480
Fig. 4C, correlation matrix DIST vs RT	Spearman’s correlation	r = 0.5666 (95% CI, 0.1208 to 0.8222); R2 = 0.3210
Fig. 4C, correlation matrix DIST vs ITI	Spearman’s correlation	r = 0.4972 (95% CI, 0.02450 to 0.7882); R2 = 0.2472
Fig. 4D, tracking distance	One-way ANOVA, Holm–Sidak’s post hoc	p = 0.03; day 1 vs day 3, p = 0.02

Results

Behavioral performance

To test our system ability to detect learning curves, we trained mice as depicted in the protocol in Figure 3. Table 2 reports all statistical analysis. In shaping phase, the average number of trials (TR) progressively increases over time for all the subjects and specifically the third day, we detect a significant increase compared with the first day. Moreover, RTs and intertrial intervals (ITIs) showed a similar trend with a significant reduction of the RT starting from the second day (Fig. 3). This indicates that already at day 2, the animals started to refine the sequence of actions necessary to trigger the stimulus and produce a response. Next, the results of the OT phase are shown in Figure 3. The average TR continued to grow until day 3 of OT. After this day, the majority of the animals performed the maximum TR permitted in each session. Both RT and ITI showed a decrease with time. Indeed, RT and ITI dropped significantly during the first 2 d and reached a plateau by the third day. We observed a significant difference in the percentage of correct responses between the first and second day. In order to assess the retention of the test over time, we tested the same mice at different time points after the end of the OT. Accuracy remained stable during all time points tested; however, RT showed a more complex pattern: with an initial decrease, compared with the last day of the OT, followed by an increase at four months. Analyzing ITI, we detected an increase of the time between two trials at 27 d that remained higher at three and four months compared with the end of the OT (Fig. 3). It is interesting to note that, since touch sensors are activated from all sides, some of the variability in timing performance could be explained by the development of different strategies to activate sensors.

Figure 3.

Behavioral performance. , Performance of the shaping phase. , OC protocol. , Performance during the OT. , Performance during recall.

Tracking analysis

In Figure 4, tracking traces, of all the mice, are shown with corresponding heatmaps, averaged across animals (on the right) or days (bottom row), showing non-uniform exploration of the OC chamber during tasks. Pixels that were not significantly explored compared with randomly resampled uniform exploration values (p > 0.05) were set to 0. The reward area was the most visited place, as shown by both the animals and session average heatmaps. In the bottom half of the arena there are two significant exploration spots at the corners, that indicate a stereotyped strategy to activate the trial (Fig. 4). Moreover, we analyzed the distance traveled by each animal inside the OC box during all the tasks. We found that, throughout the course of the shaping phase, there was a significant decrease in the total amount of distance traveled compared with the first day (Fig. 4). Conversely, during the OT phase, we detected no changes (p = 0.3672). Interestingly, we found a significant correlation between timing performances and the total distance moved during shaping (Fig. 4), this suggests that ∼25% (r2 = 0.247) of the improvement in timing performance is explained by a reduction in the distance traveled and the response of the animal. In addition, no differences in the average speed were detectable during both shaping and OT. These results imply that the reduction of the RT is due to the optimization of the psychomotor sequence in realizing the task rather than to a general increase of velocity of the animal.

Figure 4.

Tracking analysis. , Matrix of tracking traces of all animals per all days, with marginal heatmaps, showing spots of exploration significantly different from chance. Average heatmaps per each animal and per each day are presented in the last column and in the last row, respectively. , Relative exploration in the arena: reward area is the most frequently explored followed by corners of the active area and the central spot. , Correlation analysis between performance and spatial tracking. , Velocity and distance traveled during the shaping phase and the OT.

Discussion

Here, we described a fully 3D printable device that performs OC on freely moving mice while tracking the animal position in real time. We successfully trained 6 subjects, showing stereotyped learning curves that are highly reproducible across mice and reaching >70% of accuracy after 2 d of conditioning (Movie 1). This dataset demonstrate that this system is a valuable low-cost alternative to available state-of-the-art commercial devices, representing a good balance between performance, cost, and versatility. Performances detected by our system in three sessions per day (3.97 ± 0.11 trial/min with an accuracy of 84.1 ± 1.7%) are comparable with normative values detected in C57BL/6J and measured on an analogous two-alternative forced choice task performed once daily (Malkki et al., 2010). Although the LED display does not allow to design complex visual patterns required to perform image recognition and classification, visual stimulation is flexible enough to design simple tasks to test attention, learning, memory and other neuropsychological aspects of cognition (D’Ausilio, 2012; Escobar and Pérez-Herrera, 2015). The system is also easily customizable, as it is possible to add a LCD display (Movie 2) guided by extra Python libraries (e.g., Pygame or Psychopy). The overall cost of the chamber is around 160 €, but can be further substantially reduced using cheaper boards compared with AU and a Raspberry Pi. There are other low-cost alternatives for OC (Escobar and Pérez-Herrera, 2015; Francis and Kanold, 2017; O'Leary et al., 2018); however, the main strength of the present device is the high degree of reproducibility, since the model is standardized and can be downloaded to print the same structure with the same materials throughout different laboratories. Second, it can be customized in relation with specific experimental needs (e.g., very young animals). Lastly, different versions of the OC chamber can be tested and shared through on-line repositories, such as Thingiverse (https://www.thingiverse.com/) and NIH Print Exchange (https://3dprint.nih.gov/). Moreover, the OC chamber includes real time tracking of the mouse position, a feature that could be used as second phenotyping measure of anxiety or stereotyped behaviors. Additionally it allows to analyze other aspects of behavior, such as inhibitory control (Munakata et al., 2011). For example, by increasing the time required to trigger a trial, it is possible to measure impulsivity or reproduce neuropsychological tests used on humans like delayed gratification or stop signal tasks (Pinkston and Lamb, 2011; Furlong et al., 2016). It is also plausible to couple the procedure with physiological recordings in freely moving conditions such as imaging techniques (e.g., fiber photometry) and electrophysiology. Thanks to the general-purpose input/output ports (GPIO) of both AU and Raspberry Pi boards, high precision synchronization of physiological recordings with behavioral events is accurately integrated within experimental recording paradigms. The simplicity and modularity of the apparatus can be exploited as an educational tool to train students in 3D printing and coding. For these reasons, we expect that the open design of the OC chamber will be useful for teaching and research as well as for further design improvements from the international open hardware community.

A movie of a session with 30 trials during OT.

A proof of principle of LCD screen functioning inside the OC box, under the same light conditions of stimulation.