Enabling an integrated rate-temporal learning scheme on memristor.

Learning scheme is the key to the utilization of spike-based computation and the emulation of neural/synaptic behaviors toward realization of cognition. The biological observations reveal an integrated spike time- and spike rate-dependent plasticity as a function of presynaptic firing frequency. However, this integrated rate-temporal learning scheme has not been realized on any nano devices. In this paper, such scheme is successfully demonstrated on a memristor. Great robustness against the spiking rate fluctuation is achieved by waveform engineering with the aid of good analog properties exhibited by the iron oxide-based memristor. The spike-time-dependence plasticity (STDP) occurs at moderate presynaptic firing frequencies and spike-rate-dependence plasticity (SRDP) dominates other regions. This demonstration provides a novel approach in neural coding implementation, which facilitates the development of bio-inspired computing systems.

Our brain performs various cognitive tasks and outperforms the state-of-the-art von Neumann-based digital computer in many domains12. Brain inspired approach is one of the research directions to sustain the continuous performance improvement when downscaling of CMOS technology approaches its limits345. Despite the tremendous progress in VLSI technologies, it is still an insurmountable challenge to simulate our brain at the scale of 100 billion neurons and 100 trillion synapses using purely silicon-based devices678. It is widely agreed that the synapse - a biological connection between two neurons that allows information to flow from one to the other - is essential in mediating the processes of memory, learning and cognition9. A ubiquitous property of synapse is the ability to keep track of the activity history by shaping its plasticity, which is encoded via various forms of activity-dependent learning rules. Though the biological mechanism underlying the synaptic behaviors is still under debate, the identified parameters that influence synaptic plasticity including pre- and postsynaptic spiking interval101112, spiking rate1314, postsynaptic voltage1516, dendritic location1718, and postsynaptic depolarization1920 have been reported. Generally, spiking-time-dependent plasticity (STDP), often interpreted as the “first law” of synaptic plasticity, focuses on the spike timing differences between the pre- and postsynaptic neurons in modifying the synaptic weight10111221. Apart from STDP, rate-dependent plasticity learning rule, which is termed as spike-driven rate-based plasticity1322232425 or spike-rate-dependent plasticity (SRDP)26, expresses the dependence on spike frequency. Furthermore, in vivo experiments reveal that spike time- and spike rate-dependent plasticity integrate together as a function of presynaptic firing frequency1427. When presynaptic neuron fires at moderate rates (10–20 Hz), STDP learning rule mostly occurs27. Outside of the moderate frequency region, spike rate-based synaptic learning rule governs the plasticity induction and is independent of pre- and postsynaptic spiking interval101428. This integrated rate-temporal learning scheme is widely believed to play an important role in neural signal processing and information storage293031. However, to our best knowledge, such learning scheme has not been realized on any devices.

Recent advancements in memristor (also termed as memristive device)323334 have provided a strategic opportunity for advancing the development in neuromorphic engineering. This is attributed to the unique properties of the memristor including non-volatile storage, nano scale size, analog behaviors, and its ability to remember the history via the modulation of its internal state3536. It sparks a new wave of enthusiasm in developing solid-state analog synaptic devices37383940414243. In this paper, we propose a novel way to emulate the dual coding (rate and temporal) learning scheme on a memristor by customizing the presynaptic spiking waveforms. This new proposal, relying on analog properties of memristor, can be easily implemented into neural circuits.

Compared with mass storage applications, neuromorphic applications set special requirements on the memristor. It desires reliable analog properties, such as non-abrupt switching transition, continuously distributed resistance states, and repeatable behavior. In this paper, iron oxide is chosen as the resistance switching layer of memristor and its properties will be discussed first. The memristor, consisting of a sandwich-like structure of Pt/iron oxide/Pt, as shown in Fig. 1(a), was fabricated. The device size was patterned to be 0.25 μm2. The iron oxide layer (50 nm) was sputtered in a high vacuum chamber from an iron oxide compound target and the deposition temperature is below 300°C, which is compatible with semiconductor backend process specification. Based on X-ray photoelectron spectroscopy (XPS) analysis (Supplementary information), the majority compound inside the sputtered film is FeO.

Figure 1

(a) A schematic illustration of the iron oxide memristor device and cross-section view of a real device conducted in transmission electron microscopy (TEM). (b) Current-voltage (I-V) curves of memristor under multiple triangle-shapes DC sweeps. A bipolar behavior and continuous distribution of resistance states are demonstrated.

The iron oxide memristor was first investigated under DC mode. When inputting a triangle-wave shape DC voltage to memristor, the I–V curve in each sweeping cycle appears a banana-shape like hysteresis loop in both positive sweep (left plot) and negative sweep (right plot), as shown in Fig. 1(b). As the number of cycles increases, the conductance increases or decreases monotonically and consecutively. Compared to other reports of analog memristors3740, the iron oxide memristor exhibits two main differences. First, the adjacent sweeping curves just coincide with each other at the low voltage region and there is no overlapping, indicating improved data retention. This coincidence region also shows a square law relationship between current and voltage (Supplementary information), which indicates a space-charge-limited current (SCLC) conduction mechanism44. Second, there is no fluctuation or abrupt change in I–V curves during sweeping, implying a continuous distribution of resistance states.

The programming characteristics of the set and reset operation were investigated under pulse mode. When the pulse amplitude is fixed during the set operation, as shown in the left plot of Fig. 2(a), a logarithmic relationship between the changes of conductance and pulse width is observed. On the other hand, when the pulse width is fixed, it exhibits an approximate linear relationship between the pulse amplitude and the change of conductance when the pulse amplitude above the threshold, as illustrated in the right plot of Fig. 2(a). In summary, voltage amplitude has more impact on the weight change of memristor than pulse width.

Figure 2

(a) The impact of built-in conductance under varying pulse width and pulse amplitude. Left plot is under fixed positive amplitude (~1.88 V) and right plot is under fixed pulse width (1 ms). The built-in conductance was read at 0.1 V after each pulsing. (b) The relationship between threshold voltage and pulse width. Inset: threshold voltages extracted under linear fitting. (c) The decay performance of iron oxide-based memristor. (d) An illustration of repeatability of iron oxide memristor under consecutive pulse trains. Each of positive/negative pulse train consists of 15 pulses.

It is also found that the threshold voltage of the iron oxide memristor is pulse-width dependent. When varying the pulse width, the onset of conduction change is varied. By linearly fitting the measurement points, as shown in the inset of Fig. 2(b), the threshold voltages under different pulse widths can be extracted. A higher threshold voltage is accompanied by a shorter pulse width. The threshold voltages are compiled in Fig. 2(b). The figure shows an inversely linear relationship between threshold voltage and logarithm of pulse width. This indicates that if the pulse width is shrunk to nanosecond scale, a much higher voltage is required in order to move the built-in conductance, which makes the memristor robust to circuit glitches. In addition, if the memristor is heavily stressed, the memory window can be as large as 19 folds.

The memristor takes more than 4 months to decay to half of the memory window and several years to return to the starting point, as shown in the inset of Fig. 2(c). The strong retention loss reported from WOx memristor37 has not been observed in the iron oxide memristor. Based on the exponential decay function proposed by Hermann Ebbinghaus , the relative strength (S) of iron oxide memristor is fitted to be 8.9, as shown in Fig. 2(c), and the decay curve is not related to the pulse numbers. In addition, according to the report from Wickliffe C. Abraham in 2003 that “LTP (Long term potentiation) can last for hours, days or even months, and usually follows an exponential decay”45, the iron oxide-based memristor that retains the resistance for months can act as a long-term synapse in a neuromorphic circuit.

In order to form the analog memristor, a high voltage forming process is required, as illustrated in Supplementary Fig. S2. After the forming process, the memristor devices all exhibit similar memory window, despite the device size varying from 0.25 μm2 to 16 μm2, which implies a formation of conducting filament46. Besides, the non-linear I–V curve at low electric field region discussed above suggests that the memristor works under the condition of filament rupture. The condition of rupture point determines the resistance changing behavior.

Thus far, the characteristics of the iron oxide memristor have been analyzed. With proper initialization, the memory device can be repeatedly switched back and forth, as shown in Fig. 2(d). Compared to other reported memristors404748, the iron oxide-based memristor demonstrates improved controllability during programming and improved repeatability between different pulse trains. By fully utilizing these analog behaviors, the iron oxide memristor can be used to emulate the synaptic learning rules.

Unlike STDP synaptic learning rule, the SRDP synaptic learning rule has not been demonstrated using memristor. Several papers reported frequency dependence of memristor that different programming frequencies lead to much different decay curves38495051. This behaviour is used to mimic the transition of short-term memory to long-term memory. However, it cannot be used to emulate the SRDP learning rule. Because SRDP requires bidirectional changing plasticity at different frequency region - low frequency induces LTD (decrease of plasticity) and high frequency induces LTP (increase of plasticity). In order to realize this bidirectional changing behaviour, we propose a novel way by engineering the input waveforms, which not only realizes SRDP, but further integrates with STDP together as a whole to achieve a bio-plausible integrated learning scheme.

The presynaptic spike is customized by drawing inspiration from the firing behavior of biological neurons. A typical biological neural firing curve is shown in the bottom left of Fig. 3(a)52. Based on this biological firing curve, a similar shape of presynaptic spike is constructed. As shown in the bottom right of Fig. 3(a), the presynaptic spike consists of two pulses. One is a short and high-amplitude pulse, followed by a relatively wide and low-amplitude pulse in the opposite direction. Construction of both negative and positive pulses in one presynaptic spike aims to realize bidirectional weight change when the presynaptic firing frequency varies. Using a simple circuit (supplementary document), this tailored presynaptic spike can be easily realized with a normal pulse input.

Figure 3

(a) A simple illustration of neuron, synapse and neural spike. The customized spike (bottom right plot) is inspired by the biological firing curve (bottom left plot). (b) An illustration of spike waveforms at different presynaptic firing frequency. (c) An emulated of SRDP learning rule on iron oxide memristor. (d) The reported biological SRDP curve26.

In order to match the plasticity changing direction that low frequency induces long-term depression (LTD), the presynaptic spike shape used during the test is upside-down. That is, the short pulse has negative amplitude (−1.8 V), followed by the positive low amplitude (0.5 V), as shown in Fig. 3(b). This large negative amplitude (−1.8 V) decreases the memristor conductance greatly and the other half of the spike - low-amplitude positive pulse (0.5 V) has negligible impact on the memristor because it is below the threshold voltage. Thus, the overall effect of presynaptic spike decreases the conductivity, which realizes LTD at the low spike rate condition (<5 kHz). When the spike rate is above 5 kHz, the positive and negative pulses of presynaptic spikes overlap each other, thus cancelling the negative amplitude, resulting in a smaller decrease in conductance. Even higher presynaptic frequency, such as 20 kHz, further cancels the negative amplitude, as well as accumulates the positive pulses to be far above positive threshold voltage, causing a significant increase in memristor conductance. An illustration of the above discussed waveforms is shown in Fig. 3(b), using 2.5, 10 and 20 kHz presynaptic spike frequency as examples. When these presynaptic spikes are input into the iron oxide memristor, a frequency dependent plasticity behavior is demonstrated, as shown in Fig. 3(c), that low frequency of presynaptic spikes decreases the conductivity (LTD), and high frequency spikes increase the conductivity (LTP). This frequency dependent behavior matches the biological report of SRDP26, which is redrawn in Fig. 3(d).

Furthermore, the spiking-time-dependent plasticity (STDP) learning rule is also demonstrated using the same presynaptic spike trains. Similar to literature reports394041515354, a special tailored pulse train (termed as STDP pulse train) with varying voltage amplitudes is constructed in order to correlate the change of conductance with pre- and postsynaptic firing interval (Δt = t − t), as shown in Fig. 4(a). The reason of using various pulse amplitudes in constructing the STDP pulse train instead of pulse widths is due to the higher sensitivity of the pulse amplitude of the memristor. The STDP pulse train consists of 8 pulses: 4 positive pulses followed by 4 negative pulses. All pulses have the same pulse width (4 μs) and the same intervals (4 μs). The center point of STDP pulse train is aligned with the rising edge of the presynaptic spike.

Figure 4

(a) An example of pulsing scheme for the realization of STDP learning rule. (b) A schematic illustration of synapse circuit to achieve dual coding learning scheme. (c) A typical STDP learning rule emulated using iron oxide memristor at 10 kHz presynaptic firing frequency. (d) A summary of learning rule integration when varying presynaptic firing frequency. It shows that STDP learning rule only happens at the moderate frequency region and other regions are dominated by the SRDP learning rule.

Besides, inspired by the refractory period and back-propagation effect, an operational schematic of synapse is illustrated in Fig. 4(b) to realize the STDP learning rule. The refractory period refers to the biological information block period that is caused by the sodium ion channels inactivation. During this period, neuron does not fire again irrespective of incoming stimulus and resets itself to resting potential52. A switch (K1 switch) is used to realize and emulate the refractory period in our proposal. Unless the post-neuron fires, the iron oxide memristor connects the pre-synaptic neuron and post-synaptic neuron directly, enabling the information to pass through. Once the post-synaptic neuron fires, this firing information back-propagates to K1 switch and triggers it switching to STDP waveform terminal for a short period (8 μs) – refractory period. During this period, the information from presynaptic neuron will be blocked and the postsynaptic neuron will reset its potential to initial condition. At the same time, the presynaptic spike will overlap with the STDP pulses to modify the conductivity of the memristor – inducing the plasticity change.

A typical STDP is demonstrated in Fig. 4(c) at 10 kHz presynaptic frequency. Here, we assume nine-times firing of pre-synaptic neuron results in one time firing of post-synaptic neuron and the voltage drop on memristor is clamped to 2.5 V to avoid the breakdown of the memristor device. As shown in Fig. 4(c), when pre-synaptic neuron fires before post-synaptic neuron (Δt > 0), LTP is induced. Otherwise, LTD occurs. Moreover, the smaller of the pre- and post-synaptic firing interval results in a larger change of conductivity, and vice versa. However, when the pre-synaptic neuron fires at other frequency region (e.g. 2.5 kHz, 20 kHz), the STDP behavior is not observed regardless of the pre- and post-synaptic firing interval. This is due to the canceling effect between the STDP pulse train and presynaptic neural spikes (supplementary information). The frequency-dependent behaviors are summarized in Fig. 4(d), showing that STDP only happens at the moderate firing rate region (ranging from 8 kHz to 10 kHz in our test, as highlighted in Fig. 3(c)). Outside the moderate region, higher firing rates only lead to an increase of conductivity (LTP) and lower firing rate causes a decrease in conductivity (LTD). These observations are consistent with biological reports27, demonstrating a dual coding learning scheme on a single nano device. It is necessary to point out that constant inputting firing rates are used during the learning rule illustration which is to simplify the analysis. However, the input neuron spikes can be inconsistent and discrete like reality. By using the waveform generation block (supplementary information), the inconsistent and discrete spikes can be easily transformed into the tailored waveform for the learning rules implementation.

It should be noted that the response of biological neurons is not uniform, but is variable and is often modeled by a statistical distribution, i.e, Poissonian, in literatures55. Devices emulating synaptic properties need to consider the robustness against non-uniform neural stimulus. Our proposal of dual coding learning scheme is robust to the variation of the presynaptic input. When the presynaptic firing frequency is slightly varied, or even one spike is missed or is inserted owing to system instability, the learning scheme will tolerate these fluctuations and output the similar results, as long as the errors are not critical. (supplementary information).

The conductance variations from device to device also exist. For example, under the same pulse width condition (0.1 ms), the threshold voltages of most devices fall in the range of 0.9 ± 0.1 V. Such variations, we believe, would not be a great obstacle in neuromorphic implementation because the variation of synapses has also been observed in human brain which does not affect cognition5657. Currently, there are two ways to deal with such variations. Simeon Bamford et al. reported that STDP is a homeostatic process which is unsupervised and self-contained. It can reduce variations in performance caused by both mismatch in fabrication and inhomogeneity in the electronic devices58. The other way proposed by Sadique Sheik et al is a totally different way59. Rather than attempting to reduce the device mismatch, he proposed to utilize these mismatches present in VLSI chips to model biological differences, such as the variations of axonal propagation delays. In this novel approach, a certain range of mismatches and variations is desirable.

It is known that different parts of the brain use different combinations of learning mechanisms. For example, our visual recognition uses spatio-temporal based learning rule and our aural recognition uses frequency based learning rule. In the grand challenge of cognitive systems, it is necessary to emulate a wide plethora of learning mechanisms to facilitate the development of the whole spectrum of cognitive functions. To this effect, we expect that our dual coding learning scheme could provide a platform for the development of neural coding technology in hierarchical structure. On the other hand, from the engineering point of view, there arises the question of whether the sole implementation of paired-based STDP learning rule, which has been reported by several groups394041426061, is enough to build a functional neural circuit. We find there are two quandaries that are hard to deal with. First, at the condition of high frequency, the learning windows of LTP and LTD will overlap each other. The post-synaptic spike located inside this overlapping region would have difficulties in identifying the proper actions. Second, when a neuron has multiple inputs, the disturbance of alien spike from different synapse may induce an undesired synaptic weight change which is in conflict with its historical contributions. Therefore, the paired-based STDP learning rule is not enough to alleviate these quandaries and spike rate dependence should be considered. In short, the dual coding learning scheme should be highly desired in the implementation of neural circuits.

In summary, based on the good analog properties exhibited from the iron oxide-based memristor, an integrated rate-temporal learning scheme is demonstrated. This emulated learning scheme is robust to the input frequency variations as well as missing spikes. We expect that the implementation of dual coding learning scheme may greatly facilitate the development of neuromorphic circuits towards real cognition.

Author Contributions

W.H. proposed and designed the experiments, and drafted the manuscript. K.H. proposed and designed the electric circuit. N.N., K.R., G.L., R.Z. and J.P. carried out the theoretical analysis. W.H., K.H., Y.J., J.S. and R.Z. did the experiments. L.S. directed the projects and provided overall guidance throughout. All authors contributed to the results analysis and reviewed the manuscript.