Spiking neural state machine for gait frequency entrainment in a flexible modular robot.

We propose a modular architecture for neuromorphic closed-loop control based on bistable relaxation oscillator modules consisting of three spiking neurons each. Like its biological prototypes, this basic component is robust to parameter variation but can be modulated by external inputs. By combining these modules, we can construct a neural state machine capable of generating the cyclic or repetitive behaviors necessary for legged locomotion. A concrete case study for the approach is provided by a modular robot constructed from flexible plastic volumetric pixels, in which we produce a forward crawling gait entrained to the natural frequency of the robot by a minimal system of twelve neurons organized into four modules.

1 Introduction

Artificial intelligence has in recent years seen a revolution in the area of “neuromorphic computation”, the use of brain-inspired computational units which, like biological neurons, communicate with each other using discrete events called “spikes” to perform a variety of tasks [1]. These techniques can have significant advantages in terms of power consumption and the ability to make use of the temporal structure of input data [2].

It is common to derive spiking neural networks directly from deep learning approaches, either by modifying the learning technique to be applicable to a spiking neural network, or by globally converting a traditional neural net into an equivalent spiking neural network [3]. Both of these approaches have shown promise on many problems, including the generation of complex behaviors for simple robots [4-6]. In this paper, we aim to illustrate a very different approach, where spiking neurons are used as elements to design a circuit which implements a desired behavior.

The function of interest in our application is a central pattern generator, or CPG, inspired by the way in which the simplest organisms such as nematodes use neurons to implement periodic motor patterns and to make simple decisions such as retreating in response to a toxic chemical stimulus [7]. Pattern generation is a well-studied topic in computational neuroscience [8], but in robotics, central pattern generators constructed from spiking neurons are relatively rare. Instead, applications typically use the abstract concept of a CPG, but construct periodic motions from periodic primitives such as phase oscillators to produce motion commands [9]. However, an approach informed by neuroscience is worthwhile because robotic experiments in this vein can generate interesting insights about the design tradeoffs made by real neural systems [10].

Applications of this principle to robotics include a variety of open-loop spiking CPGs generating position commands for robots ranging from hexapods [11] to fish [12]. Also, certain researchers have decided to tackle the problem of proprioceptive feedback in quadrupeds [13-15]. In this paper, which extends work previously presented at the 2020 IEEE International Conference on Soft Robotics [16] using a prototype of the same robot, we will demonstrate the design and implementation of a spiking central pattern generator entrained by proprioceptive feedback to control the locomotion of a modular robot. This system serves as a case study for a simple conceptual framework which allows us to implement closed-loop robotic control through a minimal spiking neural network without recourse to black-box optimization.

The organization of this paper is as follows. Section 2 describes the basic module making up the controller as well as how such modules can be used to implement arbitrary state machines. Section 3 demonstrates that the computational properties of this module are insensitive to parameter variation yet relatively easy to modulate with the appropriate inputs. Section 4 applies these principles to generate a specific desired state machine for the control of a flexible robot. Finally, Section 5 describes the experiments that were carried out on this platform.

2 Building blocks for universal computation

From a design perspective, it can be helpful to treat neurons as asynchronous digital logic elements which represent an on state by firing a train of action potentials at a given rate, and off by remaining quiescent. This simplified model necessarily ignores many subtleties of the underlying analog neural system, and it is not Turing complete [17], but it can be used to implement neural state machines for robotic control [18].

In this section, we propose a basic module which can be used to construct such a state machine in the setting of neuromorphic computation: a three-neuron circuit which can hold state, depicted in Fig 1. This module is a neuromorphic implementation of an electronic set-reset (SR) latch, a component which holds one bit of state controlled by its two inputs, “set” (S) and “reset” (R). Like its electronic prototype, this circuit can switch between two states depending on its input: an inactive resting state representing a saved logical value of 0, and an active spiking limit cycle representing a logical 1. We can construct such a circuit by connecting two neurons, labeled E1 and E2 in the figure, in strong mutual excitation; in this configuration, whenever either neuron fires, the other will fire in response, but the system can remain in a quiescent state for any duration if neither neuron is externally induced to fire.

Fig 1

The neural latch module.

Arrowheads are circular (flat) to represent excitatory (resp. inhibitory) connections. The symmetrically connected pair of excitatory neurons E1 and E2 are connected to produce positive feedback. An excitatory input to E1 on the left activates the module, and the output is taken from E2 on the right. The inhibitory interneuron I can deactivate the module in response to an external signal which enters on the bottom left.

Technically the described behavior is not tonic firing, although the time series of spikes appears that way. Tonic firing occurs when a neuron fires repeatedly in response to a constant current or due to its own intrinsic dynamics, whereas the neurons in this module only produce single action potentials in response to presynaptic firings. In that sense, this module is an oscillator constructed from two neurons each individually incapable of oscillation. Similarly, when the module state is toggled by outside stimuli, its behavior may appear superficially similar to bursting, but the underlying dynamics are fundamentally different [19]. This mechanism of rhythmogenesis is certainly not typical for biological central pattern generation [11], but it is effective from an engineering perspective.

Biological neurons are classified broadly as either excitatory or inhibitory, and inhibitory interactions are typically short range, so to complete the circuit we introduce a local inhibitory interneuron I within the module, which we call the “reset interneuron”. This neuron, which provides the only inhibitory inputs to E1 and E2, can receive a long-range excitatory connection to function as the “reset” input in the SR latch metaphor. Weak excitatory synapses from E1 and E2 to I allow the latch to modulate its own reset: an external input just barely strong enough to trigger a reset while the latch is active will not trigger a reset while the latch is inactive, conserving the energy of the interneuron.

The activity of the three neurons in the neural module is depicted in Fig 2. The excitatory neurons E1 and E2 fire in alternation until an externally-induced firing of the reset interneuron I silences them. Note that each excitatory neuron fires only when the other provides enough synaptic current to activate it. Activity stops because the final pulse of synaptic input from E2 to E1 is curtailed by the single firing of I. The effect of this inhibition is not obvious in the plot of synaptic currents due to the slow time constant of inhibition in this model; it is visible in the shape of the tail more so than in the height of the peak. However, even this small change is sufficient to stop E1 from firing, deactivating the module.

Fig 2

Activity in the spiking neural module.

Membrane voltage and input synaptic current for each neuron are displayed as the module is deactivated by some external stimulus. The three neurons are represented by their membrane voltages, with three traces corresponding to the color code established in Fig 1: the two excitatory neurons are shown in blue and orange, while the inhibitory neuron is in green.

2.1 Breaking the digital mold

Although it can help in the conceptual design phase to view neural systems through a digital lens, such models can be overly limiting in terms of efficiency and expressiveness. Many functions which would require a very large number of logic gates to implement using binary neurons can be realized in a relatively straightforward way using the analog properties of a much smaller number of spiking neurons.

From an electrophysiological perspective, a neuron is an electrically active cell with a variety of ionic channels which maintain the membrane voltage near a resting value but can generate large spike-shaped excursions called “action potentials” in response to currents induced by input neurotransmitters. Spiking neural models are those which reproduce this excitability in the form of an excitable dynamical system.

A good intuitive model of this behavior is given by the leaky integrate-and-fire (LIF) neuron, in which the membrane voltage v has linear dynamics tending towards a fixed resting potential V, but if the collective effect of the inputs drives the membrane voltage above the peak value V, a spiking event is said to have occurred, and the membrane voltage is reset to a fixed value c.

As an example of how these dynamics may be useful, the simplest realization of a delay in a digital circuit is a series of many identical short delay elements such as inverters, whereas an analog system can implement a delay through parameter tuning without introducing new circuit elements. A small excitatory input can gradually increase the membrane voltage of a neuron until it fires, so that a sequence of many presynaptic firings at sufficiently high frequency is required before any signal is propagated to the output. In fact, it is exactly this method which we will use in Section 2.3 to ensure that the activation of each neural module propagates only gradually to other modules to which it is connected.

Instead of viewing the neural module within the digital logic metaphor, we can take the dynamical perspective and treat it as a bistable relaxation oscillator. If a neuron does not fire, it does not contribute to the positive feedback loop responsible for the oscillatory state, so the state of the module as a whole remains unchanged for subthreshold inputs. On the other hand, for inputs large enough to produce a spike, there is positive closed-loop gain, producing oscillatory behavior.

It is worth noting that in the context of continuous dynamics, the analogy to a state machine is made by a so-called “heteroclinic network”, where the active limit cycle of a module represents an attractor corresponding to a state [20]. Inputs, disturbances, or weak coupling between modules can move the system state from one attractor to another, producing a state transition along a heteroclinic trajectory.

Other dynamical properties of the neuron may also be used for computation. The morphological diversity of the real neuron seems to be used to great advantage in this context: dendritic computation and the availability of a wide array of different neurotransmitters and classes of inhibition can allow the implementation of logic gates at a finer granularity than the individual cell [7]. Likewise, it is possible for the dynamics of individual neurons to produce bistability between resting and tonic firing or between tonic firing and bursting [21]. Additionally, besides the “integrator-type” neurons which we have described conceptually, neurons also exist which are electrically resonant, responding preferentially to inputs which come at a certain frequency [19]. However, we will not be exploring these phenomena in this paper.

2.2 Modeling spiking neurons

In order to take advantage of the analog computational properties of spiking neurons, we apply the Izhikevich model together with conductance-based synapses to describe the behavior of our simulated neurons. We have chosen to adapt Izhikevich’s spiking neural model because it trades off phenomenological accuracy with efficient computation in a reasonable way [22]. This model is essentially a LIF neuron augmented with two main features: a quadratic nonlinearity to reproduce the dynamics of the biological threshold behavior, and the recovery variable u, which abstracts various slow ionic currents to provide persistent state across spikes. The model parameters are physically interpretable, and the model dynamics can be fit to observed electrophysiological data.

Although the Izhikevich model is not the simplest neural model capable of producing a bistable oscillator network [23], its accurate recapitulation of electrophysiological activity is important from a design perspective. Qualitatively similar behavior can be generated by a variety of different systems, but greater fidelity to biological prototypes helps a system designer consider constraints comparable to those faced in the evolution of biological organisms.

Once individual neurons have been modeled, the next consideration is how to connect them in networks. Large neural networks commonly employ a Dirac delta synapse, where each firing of a presynaptic neuron instantaneously increases the membrane voltage of the postsynaptic neuron. However, phase synchronization of integrator-type neurons requires synaptic activity with finite nonzero duration [24]. For this reason, we implement conductance-based synapses, where each synaptic connection is represented as a conducting channel gated by a presynaptic activation x, which increases after a firing event and gradually decays back to zero. When activated, the channel admits a current proportional to the presynaptic activation times the difference between the postsynaptic membrane potential and the synaptic reversal potential [25].

The time course of the presynaptic activation follows the classic “alpha” function observed for synaptic conductances in vivo [26]. This function is the solution to the second-order linear differential equation , with the initial conditions x(0) = 0 and , so separating into two first-order differential equations by introducing a second variable , we obtain x and y dynamics with which the Izhikevich model can be augmented to produce the following dynamical model of a single neuron subject to an input current I(t):

After equipping these neurons with the parameters described in Table 1, we can combine them into neural networks consisting of N neurons coupled to each other only through the input current I(t). The magnitude of this current is parameterized by a matrix G whose entries specify the peak input synaptic conductance triggered in the ith neuron by a firing in the jth neuron. The synaptic reversal potential V is also specified per presynaptic neuron. The resulting input I(t) to the ith neuron is therefore given by:

Table 1

Parameters for the Izhikevich neuron.

Parameter		Cell Type		Units
		RS	LTS
a	characteristic rate of u	0.03	0.03	ms−1
b	leakage conductance	-2	8	nS
c	downstroke return voltage	-50	-53	mV
d	downstroke inrush current	100	20	pA
C	membrane capacitance	100	100	pF
k	sodium channel gain	0.7	1.0	nS mV−1
Vr	resting potential	-60	-56	mV
Vt	threshold voltage	-40	-42	mV
Vn	synaptic reversal potential	0	-70	mV
Vp	action potential peak	35	20	mV
τ	synaptic time constant	5	20	ms

Finally, we generate an actuation effort command for the physical actuators using simple simulated muscle cells. Because our system is designed with invertebrate control schemes in mind, it is reasonable to use a muscle model where, unlike the electrically active vertebrate muscle cell, muscle cells do not fire action potentials. Instead, they simply provide contractile force in proportion to their depolarization [27]. The result is similar to applying an exponential low-pass filter to the spiking waveform [11], smoothing the output of the system so that changes occur only on the timescale of a few milliseconds, determined by the synaptic and membrane time constants.

2.3 Choosing model parameters

We have designed the neural latch of Fig 1 according to a simplified binary neural model, whereas the spiking dynamical neural model with which we are simulating our circuit requires all of the various parameters summarized in Table 1. The question then arises of how one should choose the values of these parameters in order to obtain some desired behavior.

We use the parameter values shown in Table 1, corresponding to well-characterized human cell types from the literature [19]. Excitatory cells are modeled as regular spiking (RS) pyramidal cells, and inhibitory cells take parameters corresponding to low-threshold spiking (LTS) inhibitory interneurons. These parameters have been chosen mainly based on their ready availability and the fact that they do not have intrinsic bursting dynamics or bistability, allowing us to implement the essential features of the module in circuitry rather than depending on exotic dynamics. However, it is fully possible to use parameters corresponding to other cell types in order to implement the same behaviors. For example, if we had used a set of parameters which produces bursting dynamics, our module might have implemented a limit cycle where the two neurons alternate bursts rather than spikes, but the same logical operations and constructions would have been available to us. As we will see later, this structural style of design leads to robust behavior in the module, in contrast with the precise tuning that would typically be necessary if all of our logical operations were implemented using such analog properties.

With these values given, only the following synaptic conductances must be tuned manually in order to implement any desired logical structure: the strong connection Gexc between the two excitatory neurons within the module, another relatively strong synapse Ginh allowing the reset interneuron I to deactivate the module, a weak synapse Grst priming the reset to fire while the module is active, a relatively weak feedforward input Gffw by which one module might gradually activate another, and a moderately strong connection Gfb by which one module can quickly deactivate another. We found empirically that these parameters can be set with only a small amount of trial-and-error fine tuning due to their wide tolerances, as we will see in Section 3.1. Furthermore, three of them are internal to the module, and so robustness in these parameters is inherited by larger networks constructed from the same building block. The values we arrived at are given in Table 2.

Table 2

Synaptic parameters for the neural latch module.

Parameter and Use		Value (nS)
Gexc	connection within the module	20
Ginh	strength of the “reset” interneuron	10
Grst	priming the reset during module activity	5
Gffw	gradual activation of other modules	10
Gfb	deactivation of other modules	10

The neuron parameters affect the behavior of the module, but to varying degree. For instance, the action potential peak V can be set almost completely arbitrarily because by the peak of an action potential, v is on a trajectory which would reach infinity in only a few milliseconds if not for the spike reset. As a result, the choice of precisely where to clip the spike does not substantially alter the trajectory [19].

Other parameters have a non-negligible effect on the period of the oscillation within the module, but for many of these, since their main effect is essentially shifting the timescale on which the neuronal dynamics play out, they have much less effect on the quantities which we want to control, such as the activation thresholds and necessary synaptic conductances. For example, increasing the after-spike reset voltage c essentially gives the dynamics of the neuron a head start along the trajectory that must be traversed between each firing, meaning that the period decreases, but because it does not affect the continuous dynamics, this has no effect at all on the process of initiating the active state.

More pertinent to activation behavior, the strength of the excitation from one neuron to another is controlled by several synaptic parameters: the synaptic time constant τ, the peak synaptic conductance Gexc, and the synaptic reversal potential V all control the shape and scale of the postsynaptic potential initiated by a firing event. A synaptic activation with a longer duration, higher peak conductance, or higher synaptic reversal potential will produce a larger or longer postsynaptic potential, and therefore cause a postsynaptic firing either more quickly or more surely.

One may imagine the neuron as a simple circuit with a single capacitance representing the cell membrane, and transmembrane conductances connecting this capacitor to voltage sources corresponding to ionic reversal potentials. In particular, the synaptic conductance connects to the reversal potential V. This view is the basis of biophysical models of neuronal dynamics (see, for example, ref. [25]), but as a mental image it can potentially be misleading because it encourages visualizing the steady state of an RC circuit, where the actual value of the conductance is unimportant; in fact, the synaptic conductance is only nonzero for a short time relative to the RC time constant, generating a small deviation in the voltage v across the capacitor. As a result, the value of the conductance actually has a substantial impact on the amplitude of the postsynaptic potential, directly affecting whether or not the membrane voltage surpasses the threshold. Beyond this minimum, there is a wide range of acceptable values for this conductance.

3 Robustness of the neural module

Biological central pattern generation is carried out by small networks of neurons which must be robust to certain types of parameter variation in order for their behavior to withstand changes in the environment, but at the same time must be sensitive to other changes so that their behavior can be modulated by other components of the nervous system [28]. Although certain organisms may utilize the redundancy of large cell populations in neural circuits to mitigate unavoidable variations in neuronal parameters [29], our constructed neural system is not redundant and so must be intrinsically robust.

In this section, we quantify the behavior of the neural latch module of Fig 1 with respect to variations in the parameters of the two excitatory neurons. First, dynamical analysis is applied to measure the maximum deviation in each parameter for which the existence of the spiking limit cycle is preserved. Next, we use a Monte Carlo approach to quantify the sensitivity of the module to simultaneous variation in all twelve parameters of both neurons.

3.1 Preservation of the spiking limit cycle

The essential function of the module is to switch between resting and spiking states in response to external input. In the absence of bounds on the strength of the input, as long as both states exist and continue to be attractive, it is always possible to switch between them. Therefore, we operationalize the question of whether a given parameter set produces a viable module by checking for the existence and stability of both attractors.

The existence of the resting state can be checked by finding a fixed point of the dynamics, where all nullclines in the system intersect; since three of the four phase variables of each neuron obey linear dynamics, a fixed point can exist only when both neurons have (u, x, y) = (b(v − V), 0, 0). Under this condition, since the synaptic activation variable x is zero, the neurons are decoupled, and the quadratic dynamics of the single remaining phase variable v have roots at v = V and . Varying the parameters b, k, V, and V produces a transcritical bifurcation when . For these parameter values, the two fixed points coincide, resulting in a single nonhyperbolic half-stable fixed point. However, for all other values of the parameters, the continuous dynamics exhibit exactly one stable node separated from the firing threshold by the unstable manifold of a saddle point, meaning that an attractive resting state always exists, although it is only marginally stable at a single pathological point in the parameter space.

Since the resting state is always attractive, only the existence of the attractive limit cycle must be verified for different parameter sets. However, it is nontrivial to prove the existence of a limit cycle, especially for a discontinuous dynamical system such as this one. Instead, we determine acceptable parameter values by numerically approximating a discrete dynamical system called the Poincaré first return map. This map is defined from an underlying continuous dynamical system by choosing a manifold in phase space known to be transversal to flow. The Poincaré map is then defined by following the dynamics from each point on to the next point where the flow intersects . Any fixed point of P lies on a periodic orbit in the original system, which is an attracting limit cycle if the fixed point is attractive. This is explained in more detail in any standard text on dynamical systems theory (e.g. chapter 10 of ref. [30]).

Our Poincaré section is the hyperplane v1 = v. As a result, the Poincaré map P takes the state vector at each firing of the first neuron to its value the next time the first neuron fires. We compute this map numerically in the attached Julia simulation code using the ODE solver library DifferentialEquations.jl [31], then find its fixed point by Picard iteration, the process of iteratively applying the map until the results converge, starting at an arbitrary point on the Poincaré section. Fixed points found in this manner are attractive, meaning that they correspond to stable limit cycles. We also checked that if the limit cycle exists, its period is more than 5 ms so that the dynamics will remain numerically stable when computed on the robot, where we must use fixed-timestep integration with Δt in the hundreds of microseconds. However, this requirement did not affect the range of acceptable parameters.

We first check for the persistence of the spiking limit cycle when the module is subject to variations in individual parameters, with both excitatory neurons modulated identically. The computed acceptable parameter ranges are given in Table 3. Parameter differences between the two neurons typically lead to asymmetries in the spiking pattern, where the two neurons take different amounts of time to activate, although this effect does not interfere with the functionality of the module. Other undesirable effects include overexcitation causing desynchronization between the two neurons, leading to a chaotic spiking attractor rather than a limit cycle. In principle the module could still perform its function under these conditions, but it is so different from our intended behavior that we consider it defective. Gexc, τ, and V have their own interesting failure mode. As the firing rate increases, for a given value of Grst, the neuron can trigger its own reset, breaking up the limit cycle. When Grst is set to zero, these parameters can be set as large as numerical stability will allow.

Table 3

Parameter ranges for the neural latch.

Param.	Min.	Nom.	Max.	Units
a	0.017	0.030	∞	ms−1
b	−6.08	−2.0	0	nS
c	−∞	−50.0	−40.3	mV
d	40.4	100	169	pA
C	68.3	100	159	pF
k	0	0.70	1.41	nS mV−1
Vr	−66.3	−60.0	−29.2	mV
Vt	−47.0	−40.0	−36.2	mV
Vp	−32.7	35.0	∞	mV
Vn	−9.7	0.0	9.7	mV
τ	3.77	5.0	7.41	ms
Gexc	16.1	20.0	31.6	nS
Grst	0.0	5.0	7.1	nS
Ginh	4.4	10.0	∞	nS

Our prior statement that it is always possible to switch between the two attractive states if they both exist is only true if outside input to the module is unconstrained. However, inhibition to a module should come only from its own reset interneuron. The result is one additional constraint: the conductance Ginh must be strong enough to move the module from the spiking limit cycle to the resting state. It is simple to verify this using the Poincaré map techniques already developed. Specifically, rather than ensuring that under the given parameter values a spiking initial condition eventually converges to the limit cycle, we ensure that if Grst is increased enough that the module triggers its own reset, the resulting synaptic activity is sufficient to break the limit cycle.

When this is the case, we already know that any outside input sufficiently strong to cause either of the neurons E1 or E2 to fire will bring the module to its spiking limit cycle. Furthermore, we have established that an outside input sufficiently strong to trigger a firing of the reset interneuron will bring the module back to its resting state. This process therefore establishes both the bistability of the system and the feasibility of switching between the two attractors in response to external input which is constrained to be excitatory.

3.2 Monte carlo analysis

The method of the previous section identifies the maximum independently acceptable deviation in each individual parameter value, but concurrent deviations may interact in nontrivial ways. For example, a lower resting voltage V or a higher threshold V makes the resting state more attractive, but this can be compensated by increasing the synaptic reversal potential V.

In order to generalize to simultaneous variations in all neuron parameters, we use a Monte Carlo approach to study fractional parameter variation. We introduce a fractional variation totaling 10% by selecting points ξ from a 11-dimensional standard Gaussian distribution and rescaling to set ∥ξ∥ = 1. We then multiply each of the parameters by 10% of the base value of the corresponding parameter, and add this value to the initial parameter vector. This method selects a random parameter variation totaling exactly 10%, but because ξ has been sampled uniformly from the surface of the unit sphere, its components are not independent: a large deviation in one direction typically corresponds to smaller deviations in other parameters.

For illustrative purposes, a few examples of the typical effect of this type of parameter variation on the behavior of the module are demonstrated in Fig 3. The membrane voltage of the two excitatory neurons during the spiking limit cycle of the unmodified module is compared to three different modules, each with all parameters of both neurons E1 and E2 randomly adjusted by 10%. In each of these cases, although the relative timing of spiking events varies substantially, the main qualitative features are identical: the two excitatory neurons alternate firing in a consistent pattern.

Fig 3

Effect of random parameter variation.

The module was simulated four times with randomized parameter values. The four plots represent the membrane voltage of the two excitatory neurons E1 and E2 in these four independent experiments. The top trace corresponds to the nominal parameter values; the lower three are the result of random modulation of all parameters of both neurons. The traces are qualitatively identical, and differ only in details of spike timing.

Although Fig 3 demonstrates three cases where random parameter variation did not affect the qualitative behavior of the module, this is not always the case. Sometimes, the variant module is not viable in that its dynamics converge globally to the resting state. Whether a module functions correctly or not depends on the parameter change which occurs. We quantified these effects with a Monte Carlo approach, where one million random variant modules were tested to approximate the probability with which an introduced 10% parameter variation causes the module to no longer function. This test was run in three separate variants: modifying only one of the two neurons led to failure in 1.3% of cases, whereas applying the same modification to both neurons led to failure in 2.8% of cases, and modifying both neurons independently led to failure in 5.4% of cases.

Variations in different parameters have different importance. Many of the parameters have little effect on the observable dynamics of the neurons when only small deviations are considered, while other parameters may be close to a threshold beyond which the module cannot function. To study this effect, we consider a point cloud of the twelve-dimensional relative deviation variables ξ. These are unitless random variables, generated as described above, which specify the relative deviation in each parameter. The points in this cloud are labeled according to whether the module continued to function in the sense of Section 3.1 when subjected to that deviation. Since this point cloud is symmetric, dimensionality reductions which revolve around spatial clustering, such as principal component analysis (PCA), are not applicable. Instead, we use our labels for linear discriminant analysis (LDA), a simple machine learning method based on the optimization problem of finding the hyperplane which best separates two label classes. In this case, we are not trying to classify our points, but we can use the hyperplane for visualization. The normal vector of this hyperplane is a basis vector in the higher-dimensional space such that the projection of the point cloud onto this axis is maximally informative.

Indeed, the point cloud is nearly linearly separable, i.e. nearly every module failure can be attributed to the component of the deviation which occurred along a single axis. In this direction, a 5% deviation was sufficient to cause a failure, whereas even 15% deviation along other axes did not have this effect. The projection of the higher-dimensional point cloud onto a two-dimensional plane is shown in Fig 4. The horizontal axis of the projection is the discriminant axis returned by LDA, but the vertical axis is arbitrary because all remaining directions are equally (un)informative.

Fig 4

Parameter variation point cloud.

A point cloud of the two-dimensional projection by LDA (described in the text) of the twelve-dimensional variable ξ representing fractional variation in parameter values (increased to 15% total for illustrative purposes) for 1000 realizations of the CPG module. Blue points represent parameter values under which the module continues to function, and orange points represent those for which a failure was detected. The orange points in the majority-blue region are a result of the projection to lower dimension, rather than of inconsistent or nondeterministic behavior.

This linear discriminant axis is aligned with changes in the threshold, resting, and reversal potential parameters V, V, and V. Sensitivity to these parameters is not necessarily significant to the system design, however, because voltages in biological systems are tightly controlled through homeostatic regulation of ionic concentrations [25]. Parameters such as conductance which are not directly regulated are typically expected to vary much more with temperature [32]; these parameters have much less effect on our system behavior.

Although the points in this projection are almost linearly separable along this axis, a few points within the larger cloud correspond to a module failure but appear to be outliers because the projection has moved them away from the border of the allowed region of parameter space. In our system, just as has been observed in computational models of a lobster digestive CPG [33], the allowed region of parameter space has nontrivial higher-dimensional geometry. In particular, the same behaviors could have been achieved using entirely different parameter values corresponding to other cell types. We observe near-perfect linear separability because our initial parameter values are relatively close to a particular failure mode, but as the size of the point cloud increases, these lower-dimensional projections become progressively less informative as they become dominated by apparent outlier points, which make up only a small fraction of the points in Fig 4.

4 Robotic case study

The basic motion primitives underlying locomotion are the periodic oscillations which drive different motor components. For example, the C. elegans locomotor system is based on a central pattern generator which produces a repeating ripple pattern down the body of the worm, propelling it forward [34]. Likewise, in many insects, each leg is associated with a CPG that produces a single step and can be modulated by higher-level control [35] as well as sensory feedback [36] to generate the full walking gait of the organism.

We can design a central pattern generation network of this type for a robotic application by combining the abstraction of asynchronous digital logic with analog effects inspired by these biological prototypes. The remainder of this paper will be focused on the design and validation of a central pattern generation network for a flexible modular robot. This robot produces a forward crawling gait, which can be viewed as a model of worm locomotion, using four linear actuators driven by a simple four-phase finite state machine.

4.1 The robot and its gait

The ideal robotic application for a minimal CPG is one whose locomotion is produced by rhythmic activity, but which does not need high-bandwidth control logic to stay upright. Past researchers have fulfilled this requirement with a variety of different statically stable robotic systems, in particular many different types of modular robots [37-40]. Likewise, the robotic application for our CPG is a modular robot constructed from volumetric pixels [41]. These “voxels” take the form of cubic octahedra, which can be interlocked using nuts and bolts to create a deformable lattice that in larger structures can function as a metamaterial [42]. Ten voxels are arranged into two layers in the configuration shown in Fig 5, with 3D-printed feet attached to four of the voxels on the bottom layer.

Fig 5

The physical robot.

A diagram of the flexible modular robot for which our spiking central pattern generator is designed, shown in both side-on (above) and top-down (below) views. The three different voxel colors correspond to the three different materials described in the text.

The robot electronics comprise four DC linear actuators and a control module containing custom motor drive electronics as well as a BeagleBone Black single-board computer. We use DC linear actuators rather than the linear servos which are common in similar applications because servos do not provide pose information to the software. Each of the four actuators is controlled through an L298 DC motor driver, which receives a motor direction signal and PWM enable input from the microcontroller. Each actuator also contains a linear potentiometer used as a voltage divider to produce a voltage proportional to the actuator position; the ADC reads a 10-bit fixed-point number equal to the actuator extension as a fraction of its stroke. The CPG network can use this information as a form of proprioceptive feedback in order to implement closed-loop control rather than relying on the position control built into a servo.

Our robot’s walking gait is generated by four distinct, symmetrical actuation phases, each of which requires the robot to extend and contract two actuators which mirror each other while the other two are held in place. First, actuator #1 extends while actuator #3 contracts. This lifts the left front foot off the ground, while the right front foot remains planted. Second, actuator #2 extends while actuator #4 contracts; this deforms the robot’s body so that the left front foot, which is no longer in contact with the ground, moves forward. During this motion, the left front foot remains in place on the ground due to the friction between the surface and the spiky 3D-printed foot, while the smooth rear feet are free to slide. Next, actuator #1 contracts while actuator #3 extends, lowering the left front foot and raising the right. Finally, the right foot is moved forwards by contracting actuator #2 while extending actuator #4.

An advantage of the modular physical design through voxels is that individual structural subunits can be swapped out for others with different material properties—for example, our robot uses three different voxel materials. The main body is constructed from the most compliant resin so that it does not present significant resistance to the actuators, while the front feet are made from relatively rigid carbon fiber for more efficient use of the deformation generated in the body. Additionally, the node on the bottom to which the feet attach is made from a slightly stiffer resin in order to transmit motion to the feet more effectively.

4.2 Development of the CPG

We directly implement a state machine which controls the actuation of the original robot; four neural latch modules connected in a ring represent a one-hot encoding of the current state. Once any of the four modules is activated, for example by an external input, it gradually activates its successor, which in turn deactivates the first module. The result is a new form of oscillation; rather than the simple limit cycle of alternating firings produced by the original module, each module undergoes regular alternation between active and inactive states. Although the dynamical mechanism is not strictly bursting, similar bursts of spikes surrounded by quiescence are produced.

If we label the four modules A–D and identify them with the four states of our state machine, the system simply transitions through the states in alphabetical order. This corresponds to the four actuation phases described previously, where at any given time during the operation of the robot, one of the four actuators is expanding while another is contracting. At the same time, the other two actuators remain fixed in place in the absence of new motor input due to the mechanical properties of the worm gear rack-and-pinion actuation. As a result, with a state machine that cycles through four states, the desired pattern of muscle actuation is generated simply by having each state excite the corresponding extensor and the opposite flexor. The result is the full network depicted in Fig 6.

Fig 6

The full CPG network.

The system consists of four individual modules A–D connected in a ring topology. Each module is associated with one flexor (F) and one extensor (E) among the 8 muscle cells which directly control the DC actuators. The pairs of muscle cells are labeled 1–4 in correspondence with the physical actuators. Note that the depicted connections between modules are actually bidirectional—a given module activates its successor while activating the inhibitory interneuron of its predecessor.

This network implements the correct state machine, but the speed with which it switches between states is parameter-dependent and much too fast to produce the desired gait. The inertia and damping inherent to the mechanical construction of the robot mean that the actuators simply oscillate in place without producing any useful forward motion. For this reason, it becomes necessary to use feedback to modulate the speed with which the network switches between states. One can imagine that, as in C. elegans [34], the neurons of our CPG are equipped with strain-sensitive neurites which create an inhibitory leakage current in proportion to the deviation of the preceding module’s actuator from its target position. This prevents the initiation of the next state when the current state has not yet achieved its intended actuator position.

In order to describe the proprioceptive feedback Iprop, to the ith module due to this linear feedback current, we introduce the actuator position variables z and z representing the relative extension of the previous actuator and its opposite counterpart as numbers between 0 and 1. Additionally, there is a feedback constant k which must be tuned by a bit of trial and error but which depends only on the neuron parameters, not the physical properties of the robot. In our application, a value k = 25 pA was effective. The resulting feedback current is given by:

The behavior of the open-loop and closed-loop networks are contrasted in Fig 7. Both network configurations produce the same pattern of alternation between states as well as the same pattern of actuation, but the short duration for which each state remains active in the open-loop network means that the actuators only extend a very short distance before being forced to contract again. Note that this demonstration was performed in simulation for clarity; the depicted actuator position is derived from treating the actuator kinetics as ideal linear damped masses. In the realistic case, the difference is more extreme because frictional losses cause the open-loop excursion of the actuators to be significantly less than pictured here. As a result, the robot under open-loop control moves extremely inefficiently or not at all.

Fig 7

Open vs. closed loop.

Comparison in simulation between the open-loop and closed-loop oscillatory behavior of the designed neural network. The top four traces represent overall neural activity in each of the four modules during the two experimental conditions, and the lower four traces represent the extension of the actuators.

5 Physical experiment

To this point, all description of the full spiking neural network depicted in Fig 6 has been simulated, qualitative, or both. In order to provide a concrete physical demonstration of the efficacy of our approach, in this section we describe the experiments which were carried out in order to validate the physical robotic platform just described.

During all of these physical experiments, we recorded the full state of the neural network as well as the commanded actuator effort and outputs of the proprioceptive sensors. This enables us to directly correlate the activity of the neural network with the physical behavior of the robot, as in Fig 8.

Fig 8

Robot gait time-lapse.

A time-lapse of the motion of the robot’s front feet, overlaid on top of an abstract representation of the neural activity necessary to produce the depicted motion. The gray traces are overlays of the membrane voltage of all three neurons in each module, which disappear into each other due to the long timescale, and the four colored traces represent the extension of the four actuators. The four photographic insets correspond to the four phases of actuation described in the text. First, the right foot is raised above the ground as it moves forward. Next, the right foot comes down and both feet are briefly planted. Then, the left foot is raised and begins to move forward. Finally, the left foot is lowered to the ground again, and the cycle continues.

In order to quantify the physical movement of our robot, we implemented a simple motion capture system through color-based particle tracking, with three magenta 3D-printed markers affixed to the top of the robot. We assume that once the camera is calibrated to minimize distortion, position in the image plane is proportional to physical position in the ground plane. This assumption allows us to roughly calibrate the system using the fact that the distance between the motion capture markers while the robot is in a neutral pose is determined by the known geometry of the voxels.

An example of the type of data which can be captured by this system is presented in Fig 9, where the motion of the markers during forward locomotion is plotted on top of an image of the robot’s position at the end of the recording. The physical position of the robot during this behavior is shown in Fig 10; with this motion-tracking data, we can approximate the robot’s forward speed to be 3 cm/min, mainly limited by the slew rate of the actuators.

Fig 9

Overhead motion capture.

Motion capture traces demonstrating the movement in the horizontal plane (from right to left) of three plastic motion capture markers affixed to the top of the robot. White traces represent the position of the purple markers over approximately five minutes. The oscillatory patterns in the white traces are not noise in the motion capture system, but rather represent the cyclic trajectory traced out by the voxel nodes.

Fig 10

Forward motion of the robot.

The forward displacement of the three motion capture markers from their starting positions during the motion displayed schematically in Fig 9.

Next, we implemented a variation on our controller which is capable of reversing direction in response to an aversive stimulus. In a simple network like this, the easiest way to implement two different locomotive directions is the route taken by C. elegans: introducing what amounts to two parallel copies of the same CPG network [34], one of which is active during forward movement and one of which is active when the robot is moving backward. We then introduce a single neuron responsible for detecting the presence of some aversive stimulus and signaling that the forward network should be deactivated and the reverse network activated.

The result is that as soon as an aversive stimulus is detected during forward motion, the robot reverses direction and begins to move backward instead. In our experiments, the “aversive stimulus” of choice occurred at an arbitrary time, but it would be simple to replace this behavior with something more reactive, such as a physical bumper to detect obstacles. The position of the robot over time during this experiment is shown in Fig 11.

Fig 11

Reflexive reversal of the robot.

The forward displacement of the three motion capture markers from their starting positions during the reversal reflex experiment.

6 Conclusion

We have described a simple approach to the design of spiking neural networks for robotic control. By taking advantage of the fact that the behavior of neurons can be approximated by a simple binary model we can design a connectome capable of producing the desired behavior. A case study was provided by a flexible modular robot, where we applied this approach to produce a tripod-stable gait which can be modulated by sensory feedback or external inputs much like a biological central pattern generator.

In the future, we are interested in developing a method to efficiently characterize the geometry of the extended region of parameter space in which the designed neural module continues to function. This would allow a more thorough approach to the question of robustness vs. modulation, as the acceptable region of parameter space is clearly significantly larger and more irregularly shaped than the region studied here. It would also be of interest to more rigorously study the question of how single-module robustness translates to the full system.

Additionally, in the real-world evolution of central pattern generation networks, although the simple underlying architecture tends to be evolutionarily conserved, evolution tends to provide a great deal of variety in the ways that this architecture can be modulated by higher-level control [43]. We seek to explore such additional forms of modulation in future work; for example, we may introduce a higher-level learned or evolved network which interacts with the world by modulating or controlling an underlying conserved central pattern generation network.

Simulation code.

A Jupyter notebook containing the Julia code which implements our simulations as well as the mathematical results of Section 3.

(IPYNB)

Click here for additional data file.

Forward locomotion.

A video of the robot performing forward locomotion, from which the motion capture data in Figs 9 and 10 were computed.

(MOV)

Click here for additional data file.

Backward locomotion.

A video of the robot performing backward locomotion, from which the position data depicted in Fig 11 were computed.

(MOV)

Click here for additional data file.

Side view.

Side view of the robot taking a few steps, used to generate Fig 8.

(MOV)

Click here for additional data file.

2 Sep 2020

PONE-D-20-21490

Spiking neural state machine for gait frequency entrainment in a flexible modular robot

PLOS ONE

Dear Dr. Spaeth,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The manuscript requires revision to provide details of the model and its justification, and should be put into the context of the contemporary literature.

The introduction makes it seem that computation with spiking neural networks is a novel concept (by contrasting it to ANNs in the second paragraph of the introduction but not mentioning prior literature) developed here. - This needs major rephrasing.

The mechanisms of rhythmogenesis in the presented controller is seemingly based on temporal summation of synaptic inputs and the individual modular building blocks are not intrinsically rhythmogenic. While this is a possible mechanism for the generation of rhythmicity, there are many others and it is not the most likely one. In mammals, for example, the view is that each limb-specific circuit is capable to produce a rhythm itself (since a rhythm can be elicited in the hemicord) and slow persistent ion channels likely underly rhythm genesis. The chosen mechanism has to be discussed in connection with the literature.

The article refers to the individual modules as oscillator modules or CPG modules and even CPG networks, which is clearly misleading. The two types of oscillations described in the paper, tonic neural firing vs. neural bursting, should be clearly distinguished in the text.

Figures showing neural activities should include the synaptic currents to illustrate the mechanisms of rhythmogensis (Figs 2, 7).

It is not clear why the robustness and sensitivity of only the three-cell building block is analyzed.

How function of the latch-circuit was operationalized for the sensitivity/discriminant analysis? Was only the presence of a limit-cycle (> 5ms) used= What about the ability to change states using external inputs?

Results are mostly described qualitatively and lack quantitative detail (e.g., what are the axes in Fig 4).

The results should be presented in detail so that they would be reproducible. Parameters used in Fig 3 are not given etc. All parameters should be clearly specified and easy to relate to the described results. Furthermore, the source code for all simulations (including all figures) should be published with the paper (e.g. GitHub, ModelDB).

Introduction: Neuromorphic computation is introduced as using spikes. In the next paragraph neuromorphic machine learning is equated to artificial neural networks. This is confusing.

Line 123: “model parameters have biophysical meaning” - This is a bit of a stretch, esp. when compared to Hodgkin-Huxley type models.

Lines 169-172: The selected model parameterizations taken from Izhikevich should be explained in more detail. What is their spiking patterns. Why were they selected.

Line 203ff: This is an odd analogy. Especially since that would relate to the original non-leaky integrate and fire neuron. While the neuron at hand is the extension of the leaky version.

Fig 3: It is not clear what the authors try to illustrate. At the very least the exact parameters used should be indicated.

Please submit your revised manuscript by Oct 17 2020 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: http://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols

We look forward to receiving your revised manuscript.

Kind regards,

Gennady Cymbalyuk, Ph.D.

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Thank you for stating the following in the Financial Disclosure section:

"This research was supported by a grant made to the Braingeneers research group by

Schmidt Family Futures.

The funders had no role in study design, data collection and analysis, decision to

publish, or preparation of the manuscript."

We note that you received funding from a commercial source: "Schmidt Family Futures"

Please provide an amended Competing Interests Statement that explicitly states this commercial funder, along with any other relevant declarations relating to employment, consultancy, patents, products in development, marketed products, etc.

Within this Competing Interests Statement, please confirm that this does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests).  If there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include your amended Competing Interests Statement within your cover letter. We will change the online submission form on your behalf.

Please know it is PLOS ONE policy for corresponding authors to declare, on behalf of all authors, all potential competing interests for the purposes of transparency. PLOS defines a competing interest as anything that interferes with, or could reasonably be perceived as interfering with, the full and objective presentation, peer review, editorial decision-making, or publication of research or non-research articles submitted to one of the journals. Competing interests can be financial or non-financial, professional, or personal. Competing interests can arise in relationship to an organization or another person. Please follow this link to our website for more details on competing interests: http://journals.plos.org/plosone/s/competing-interests

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

Reviewer #2: Partly

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: N/A

Reviewer #2: No

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: Yes

Reviewer #2: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

Reviewer #2: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: This is a very interesting paper reporting ‘modular architecture for neuromorphic closed-loop control based on bistable relaxation oscillator modules’. Authors have designed ‘neural state machine’ controller that was used to control legged robot locomotion. This controller consists of 4 modules organized in circle. Each module represents 3 neuron unit burst generator, two neurons are reciprocally activating each other thus producing spike oscillations while third neuron regulating start and stop of spiking trains by inhibiting both these neurons. A robot constructed by authors has 4 legs controlled by 4 linear actuators. Each linear actuator is controlled by two opposite CPG modules in the ring providing command to flex or extend robot leg.

This is a very well performed implication of Unit-Burst-Generator hypothesis for CPG organization by Grillner to a robot and meticulously analyzed sensitivity of the model parameters shows robustness of proposed controller.

The paper describes that robot models legged locomotion, that is much more complex behavior than just flexion/extension of legs. It looks like model of worm locomotion for me.

The paper would be stronger if the authors could avoid using electrical engineer jargon. Examples: line 17 ‘converted ANNs’; line 52 ‘SR latch’.

Minor

Lines 30-31: For the sake of completeness, the discussion of the bistability in neuron models should include also the study by Dashevskiy T, Cymbalyuk G.Front Comput Neurosci. 2018, Barnett W, O'Brien G, Cymbalyuk G.J Neurosci Methods. 2013, Malashchenko T, Shilnikov A, Cymbalyuk G.PLoS One. 2011;6(7), Shilnikov A, Calabrese RL, Cymbalyuk G.Phys Rev E Stat Nonlin Soft Matter Phys. 2005

Reviewer #2: Spaeth et al. outline an approach to construct a state-machine using modules of spiking neural circuits. They describe bifurcations and analyze the robustness of SR-latch-like circuits. Then they assemble them into a state-machine and use it to control a four-legged crawling robot. They argue that these circuits resemble those of a CPG in the biological system.

The paper is well written and generally well thought out but also has some major issues: it is not properly put in the context of prior literature and lacks crucial detail in the description of the results. My detailed comments are listed in the following:

1 There is a large body of literature describing models of locomotor CPGs using populations of spiking neurons, such as the work of Eve Marder and Ilya Rybak and others. These warrant discussion. As the paper currently reads, a naive reader could think that the use of spiking neurons for CPG models is a novelty of the paper.

1a Further, there are several examples of spiking neural networks used for the control of robots (a simple google scholar search will reveal several).

1b The biggest issue of the paper is, that the introduction makes it seem that computation with spiking neural networks is a novel concept (by contrasting it to ANNs in the second paragraph of the introduction but not mentioning prior literature) developed here. - This needs major rephrasing.

2 The mechanisms of rhythmogenesis in the presented controller is seemingly based on temporal summation of synaptic inputs and the individual modular building blocks are not intrinsically rhythmogenic. While this is a possible mechanism for the generation of rhythmicity, there are many others and it is not the most likely one. In mammals, for example, the view is that each limb-specific circuit is capable to produce a rhythm itself (since a rhythm can be elicited in the hemicord) and slow persistent ion channels likely underly rhythm genesis. The chosen mechanism has to be discussed in connection with the literature.

2a The article refers to the individual modules as oscillator modules or CPG modules and even CPG networks, which is clearly misleading. The two types of oscillations described in the paper, tonic neural firing vs. neural bursting, should be clearly distinguished in the text.

2b Figures showing neural activities should include the synaptic currents to illustrate the mechanisms of rhythmogensis (Figs 2, 7).

3 It is not clear why the robustness and sensitivity of only the three-cell building block is analyzed. The robustness of the final network would be of much greater interest, especially since the single module seems to lack the ability to generate bursts.

3a It is not exactly clear how function of the latch-circuit was operationalized for the sensitivity/discriminant analysis. Was only the presence of a limit-cycle (> 5ms) used= What about the ability to change states using external inputs?

3b Results are mostly described qualitatively and lack quantitative detail (e.g., what are the axes in Fig 4).

4 The results are unlikely to be reproducible. For example, I couldn’t find the parameters for the synaptic conductances between the modules, parameters used in Fig 3 are not given etc. All parameters should be clearly specified and easy to relate to the described results. Furthermore, the source code for all simulations (including all figures) should be published with the paper (e.g. GitHub, ModelDB).

Some minor comments:

Introduction: Neuromorphic computation is introduced as using spikes. In the next paragraph neuromorphic machine learning is equated to artificial neural networks. This is confusing.

Line 123: “model parameters have biophysical meaning” - This is a bit of a stretch, esp. when compared to Hodgkin-Huxley type models.

Lines 169-172: The selected model parameterizations taken from Izhikevich should be explained in more detail. What is their spiking patterns. Why were they selected.

Line 203ff: This is an odd analogy. Especially since that would relate to the original non-leaky integrate and fire neuron. While the neuron at hand is the extension of the leaky version.

Fig 3: It is not clear to me what the authors try to illustrate. At the very least the exact parameters used should be indicated.

In summary, the paper has some value but needs major revision. Mainly the objectives of the paper have to be discussed in the context of current literature (spiking neural networks are not a novelty, neither for engineering purposes nor for models of CPGs; the novelty is the modular latch-like structure) and the rigor in the presentation of the model and analysis results has to be improved.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

Reviewer #2: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

18 Sep 2020

The text in general has been rewritten and substantially clarified, in order to better justify its position in the context of the literature as well as to clarify many of the excellent points made by the reviewers. Additionally, a new SI file has been included giving all of the simulation code in Jupyter notebook format.

The specific changes which have been made are addressed in more detail in the uploaded review response table, `responses.pdf` of our resubmission.

Submitted filename: responses.pdf

Click here for additional data file.

23 Sep 2020

Spiking neural state machine for gait frequency entrainment in a flexible modular robot

PONE-D-20-21490R1

Dear Dr. Spaeth,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Gennady Cymbalyuk, Ph.D.

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. If the authors have adequately addressed your comments raised in a previous round of review and you feel that this manuscript is now acceptable for publication, you may indicate that here to bypass the “Comments to the Author” section, enter your conflict of interest statement in the “Confidential to Editor” section, and submit your "Accept" recommendation.

Reviewer #2: All comments have been addressed

**********

2. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #2: Yes

**********

3. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #2: Yes

**********

4. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #2: Yes

**********

5. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #2: Yes

**********

6. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #2: The revised version is a clear improvement of the initial submission. The authors addressed all concerns raised by me and the other reviewer. I would like to congratulate the authors for their work and recommend this revised version of the article for publication.

**********

7. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #2: Yes: Simon M. Danner

25 Sep 2020

PONE-D-20-21490R1

Spiking neural state machine for gait frequency entrainment in a flexible modular robot

Dear Dr. Spaeth:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Gennady Cymbalyuk

Academic Editor

PLOS ONE