1

My question is about spiking neural networks. Input of a typical spiking neuron is usually some floating point value, representing its inflow current, typically expressed in mA or similar units, like in a following simple example:

static const float 
      dt = 1.0/1000,  // sampling period
      gL = 0.999,     // leak conductance
      vT = 30.0;      // spiking voltage threshold
float mV = 0;         // membrane voltage

// Leaky integrate-and-fire neuron model step
bool step_lif_neuron(float I) { // given input current "I", returns "true" if neuron had spiked
    mV += (I - mV*gL)*dt;
    if( mV > vT ) { // reset? heaviside function is non-differentiable and discontinuous
        mV = 0;
        return true;
    }
    return false;
}

That is fine, if its purpose is to determine relation of input image to some class, or to turn motor or lamp on or off.. But here comes principal problem: a model does not describe neuron interconnection. We cannot connect a neuron to next neuron, as it usually happens inside brain.

How does one convert a bool isSpiked value of preceeding neuron to float I input value of next neuron?

cheersmate
  • 2,385
  • 4
  • 19
  • 32
xakepp35
  • 2,878
  • 7
  • 26
  • 54

1 Answers1

2

This isn't a typical SO question, but here's the answer.

Of course your model doesn't answer your question as it is a neuron model. For connections (synapses in the brain, or elsewhere), you need a model for synapses. In biology, a presynaptic spike (i.e. an "input spike" to a synapse) causes a time-dependent change of the postsynaptic membrane conductance. The shape of this conductance change in biological approximately has a so-called double exponential shape:

where the presynaptic spike occured at time 0.

This conductance change leads to a (time-dependent) current into the postsynaptic neuron (i.e. the neuron receiving the input). For simplicity, many models model the input current directly. The common shapes are

  • double exponential (realistic)
  • alpha (similar to double exponential)
  • exponential (simpler and still captures the most important property)
  • rectangular (simpler, and convenient for theoretical models)
  • delta shaped (simplest, just a single pulse for one time step).

Here's a comparison scaled for same height at max:

and scaled for same overall current (so integral over the time course):

So how does a spike lead to an input current in another neuron in spiking NN models?

Assuming you model currents directly, you need to make a choice of the time course of the current which you want to use in your model. Then, every time a neuron spikes, you inject a current of the shape you chose into the connected neuron.

As an example, using exponential currents: the postsynaptic neuron has a variable I_syn which gives the synaptic input, each time a presynaptic neuron spikes, it is incremented by the weight of the connection, in all other time steps it decays exponentially with time constant of the synapse (the decay of the exponential).

Pseudocode:

// processing at time step t
I_syn *= exp(-delta_t / tau_synapse)  // delta_t is your simulation time step

foreach presynaptic_spike of neuron j:
   I_syn += weight_of_connection(j)

The topic isn't answered with a plot or two, or with a single equation. I just wanted to point out the main concepts. You can find more details the Compuational Neuroscience textbook of your choice, e.g. in Gerstner's Neuronal Dynamics (which is available via website).

cheersmate
  • 2,385
  • 4
  • 19
  • 32
  • I thought that input current is usually genetated via "dot product" : `I_syn=0; foreach(j): I_syn += X(J)*W(j); `, No? But question was about `X(J)`. I even thought about binarizing it, in sense that "if X(J) is really bool, then we would just operate with adding weight or not adding, as some input current". That would be a() = dirac delta, as you stated. – xakepp35 Oct 24 '18 at 00:53
  • The dot product `X·W` what you do in ANNs (so non-spiking neural networks). You can do the same in spiking networks when using delta-shaped currents (as you stated), or also for exponential currents (if you implement them via traces as in the code I showed). – cheersmate Oct 24 '18 at 06:11
  • If you try to run your code in imagination the very first statement would lead to undefined behaviour: `I_syn *= something`, but content of `I_syn` is undefined, It may be junk or some big value, so we almost definetely would get `NaN` there.. How do you initialize it? What is there in beginning? Also, what is `tau_synapse`? Is it const? If it is constant and `dt` is constant then we dont even need to calculate expensive `exp()` – xakepp35 Oct 24 '18 at 09:12
  • That's why it's pseudocode. `I_syn` is usually zero at the beginning of the simulation (which means you assume there were no spikes before the start), `tau_synapse` is the decay of the exponential, see Gerstner's book for the range of possible values depending on the type of synapse, and of course you can precompute the decay. – cheersmate Oct 24 '18 at 09:20
  • So should it work like "friction", assuming current = velocity(in kinematic motion): `v *= 0.99; v += a*dt;`? I understand correctly? – xakepp35 Oct 24 '18 at 10:10