Neuron Modules

Abstract Neuron

Abstract neuron base class.

class lava.lib.dl.slayer.neuron.base.Neuron(threshold, tau_grad=1, scale_grad=1, p_scale=1, w_scale=1, s_scale=1, norm=None, dropout=None, persistent_state=False, shared_param=True, requires_grad=True, complex=False)

Bases: Module

This is an abstract class that governs the minimal basic functionality of a neuron object.

Parameters:

  • threshold (float or torch tensor) – neuron threshold.

  • tau_grad (float, optional) – controls the relaxation of spike function gradient. It determines the scope of voltage/state around the neuron threshold that effectively contributes to the error. Default is 1

  • scale_grad (float, optional) – controls the scale of spike function gradient. It controls the gradient flow across layers. It should be increased if there is vanishing gradient and increased if there is exploding gradient. Default is 1

  • p_scale (int, optional) – scaling factor of neuron parameter. Default is 1

  • w_scale (int, optional) – scaling factor for dendrite input (weighted spikes). It’s good to compute synaptic operations and its gradients on smaller range. w_scale scales down the synaptic weights for quantization. The actual synaptic weights must be descaled. Default is 1

  • s_scale (int, optional) – scaling factor for neuron states. The fixed percision neuron states are scaled down by s_scale so that they are in a reasonable range for gradient flow. Default is 1

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Default is None

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no dropout. Default is None

  • persistent_state (bool, optional) – flag to enable/disable persitent state between iterations. Default is False

  • shared_param (bool, optional) – flag to enable/disable shared parameter for neuron group. If False, idividual parameters are assigned on a per-channel basis. Default is True

  • requires_grad (bool, optional) – flag to enable/disable learnable neuron decays. Default is True

  • complex (bool, optional) – flag to indicate real or complex neuron. Defaul

Neuron Attributes

threshold

Gradient Attributes

  • tau_grad

  • scale_grad

Scaling Atrributes

  • p_scale

  • w_scale

  • s_scale

Normalization Attributes

  • norm

Dropout Attributes

  • dropout

State Attributes

  • persistent_state

Group Attributes

  • shared_param

  • complex

Debug Attributes
  • debugbool

    It is False by default. There shall be no constructor access to this flag. If desired, it should be explicitly set.

clamp()
property device
quantize_8bit(weight, descale=False)

Quantization method for 8 bit equivalent input when descaled. This should be linked with synapse instance.

Parameters:

  • weight (torch.tensor) – synaptic weight.

  • descale (Bool) – flag to scale/descale the weight (Default value = False)

Returns:

quantized weight.

Return type:

torch.tensor

Examples

It can be used like a normal function. But the intended use is as follows

>>> synapse.pre_hook_fx = neuron.quantize_8bit
property threshold
property v_th_mant

Get voltage-threshold-mantessa parameter.

property weight_exponent

Get weight exponent.

CUrrent BAsed Leaky Integrate and Fire (CUBA) Neuron

CUBA neuron model.

class lava.lib.dl.slayer.neuron.cuba.Neuron(threshold, current_decay, voltage_decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False)

Bases: Neuron

This is the implementation of Loihi CUBA neuron.

u[t] &= (1 - \alpha_u)\,u[t-1] + x[t] \ v[t] &= (1 - \alpha_v)\,v[t-1] + u[t] + \text{bias} \ s[t] &= v[t] \geq \vartheta \ v[t] &= v[t]\,(1-s[t])

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • current_decay (float or tuple) – the fraction of current decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • voltage_decay (float or tuple) – the fraction of voltage decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1<<6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_current_decay

The compartment current decay parameter to be used for configuring Loihi hardware.

property cx_voltage_decay

The compartment voltage decay parameter to be used for configuring Loihi hardware.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to an input. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (torch tensor) – Input tensor.

Returns:

  • torch tensor – current response of the neuron.

  • torch tensor – voltage response of the neuron.

forward(input)

Computes the full response of the neuron instance to an input. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (torch tensor) – Input tensor.

Returns:

spike response of the neuron.

Return type:

torch tensor

property ref_delay

Refractory delay.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(voltage)

Extracts spike points from the voltage timeseries. It assumes the reset dynamics is already applied.

Parameters:

voltage (torch tensor) – neuron voltage dynamics

Returns:

spike output

Return type:

torch tensor

lava.lib.dl.slayer.neuron.cuba.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Resonate and Fire (R&F) Neuron

Resonate and Fire neuron.

class lava.lib.dl.slayer.neuron.rf.Neuron(threshold, period, decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False, log_init=True)

Bases: Neuron

This is the implementation of RF neuron.

\mathfrak{Re}(z[t]) &= (1-\alpha)(\cos\phi\ \mathfrak{Re}(z[t-1]) - \sin\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Re}(x[t]) + \text{real bias} \ \mathfrak{Im}(z[t]) &= (1-\alpha)(\sin\phi\ \mathfrak{Re}(z[t-1]) + \cos\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Im}(x[t]) + \text{imag bias} \ s[t] &= |z[t]| \geq \vartheta \text{ and } \arg(z[t])=0

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • period (float or tuple) – period of the neuron. If shared_param is False, then it can be specified as a tuple (min_period, max_period).

  • decay (float or tuple) – decay factor of the neuron. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

  • log_init (bool, optional) – if True, initialized the natural frequency in log spaced range. Default is True.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_cos_decay

The compartment cos decay parameter to be used for configuration.

property cx_sin_decay

The compartment sin decay parameter to be used for configuration.

property decay

The decay parameter of the neuron.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically. It is essentially a resonator dynamics.

Parameters:

input (tuple of torch tensors) – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

  • torch tensor – real response of the neuron.

  • torch tensor – imaginary response of the neuron.

forward(input)

Computes the full response of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

spike response of the neuron.

Return type:

torch tensor

property frequency

The frequency of neuron oscillation.

property lam

The lambda parameter of the neuron.

property period

The period of the neuron oscillation.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(real, imag)

Extracts spike points from the real and imaginary states.

Parameters:

  • real (torch tensor) – real state

  • imag (torch tensor) – imaginary state

Returns:

spike output

Return type:

torch tensor

lava.lib.dl.slayer.neuron.rf.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Resonate and Fire Izhikevich (R&F Iz) Neuron

Resonate and Fire Izhikevich neuron.

class lava.lib.dl.slayer.neuron.rf_iz.Neuron(threshold, period, decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False, log_init=True)

Bases: Neuron

This is the implementation of RF Izhikevich neuron.

\mathfrak{Re}(z[t]) &= (1-\alpha)(\cos\phi\ \mathfrak{Re}(z[t-1]) - \sin\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Re}(x[t]) + \text{real bias} \ \mathfrak{Im}(z[t]) &= (1-\alpha)(\sin\phi\ \mathfrak{Re}(z[t-1]) + \cos\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Im}(x[t]) + \text{imag bias}\ s[t] &= \mathfrak{Im}(z[t]) \geq \vartheta \ \mathfrak{Re}(z[t]) &= \mathfrak{Re}(z[t])\,(1-s[t])

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • period (float or tuple) – period of the neuron. If shared_param is False, then it can be specified as a tuple (min_period, max_period).

  • decay (float or tuple) – decay factor of the neuron. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

  • log_init (bool, optional) – if True, initialized the natural frequency in log spaced range. Default is True.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_cos_decay

The compartment cos decay parameter to be used for configuration.

property cx_sin_decay

The compartment sin decay parameter to be used for configuration.

property decay

The decay parameter of the neuron.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically. It is essentially a resonator dynamics with Izhikevich reset.

Parameters:

input (tuple of torch tensors) – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

  • torch tensor – real response of the neuron.

  • torch tensor – imaginary response of the neuron.

forward(input)

Computes the full response of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

spike response of the neuron.

Return type:

torch tensor

property frequency

The frequency of neuron oscillation.

property lam

The lambda parameter of the neuron.

property period

The period of the neuron oscillation.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(real, imag)

Extracts spike points from the real and imaginary states.

Parameters:

  • real (torch tensor) – real state

  • imag (torch tensor) – imaginary state

Returns:

spike output

Return type:

torch tensor

lava.lib.dl.slayer.neuron.rf_iz.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Adaptive Leaky Integrate and Fire (ALIF) Neuron

Adaptive Leaky Integrate and Fire neuron.

class lava.lib.dl.slayer.neuron.alif.Neuron(threshold, threshold_step, current_decay, voltage_decay, threshold_decay, refractory_decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False)

Bases: Neuron

This is the implementation of Adaptive LIF neuron.

u[t] &= (1-\alpha_u)\,u[t-1] + x[t] + \text{bias} \ v[t] &= (1-\alpha_v)\,v[t-1] + u[t] \ \vartheta[t] &= (1-\alpha_{\vartheta})\,(\vartheta[t-1] - \vartheta_0) + \vartheta_0 \ r[t] &= (1-\alpha_r)\,r[t-1] \ s[t] &= (v[t] - r[t]) \geq \vartheta[t] \ r[t] &= r[t] + 2\,\vartheta[t] \ \vartheta[t] &= \vartheta[t] + \vartheta_{\text{step}}

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – base neuron threshold.

  • threshold_step (float) – the increase in threshold after spike.

  • current_decay (float or tuple) – the fraction of current decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • voltage_decay (float or tuple) – the fraction of voltage decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • threshold_decay (float or tuple) – the fraction of threshold decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • refractory_decay (float or tuple) – the fraction of refractory decay per time step. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_current_decay

The compartment current decay parameter to be used for configuring Loihi hardware.

property cx_refractory_decay

The compartment refractory decay parameter to be used for configuring Loihi hardware.

property cx_threshold_decay

The compartment threshold decay parameter to be used for configuring Loihi hardware.

property cx_voltage_decay

The compartment voltage decay parameter to be used for configuring Loihi hardware.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to an input. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (torch tensor) – Input tensor.

Returns:

  • torch tensor – current response of the neuron.

  • torch tensor – voltage response of the neuron.

  • torch tensor – adaptive threshold of the neuron.

  • torch tensor – refractory response of the neuorn.

forward(input)

Computes the full response of the neuron instance to an input. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (torch tensor) – Input tensor.

Returns:

spike response of the neuron.

Return type:

torch tensor

property ref_delay

Refractory delay.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(voltage, threshold, refractory)

Extracts spike points from the voltage timeseries.

Parameters:

  • voltage (torch tensor) – neuron voltage dynamics of the neuron.

  • threshold (torch tensor) – threshold dynamics of the neuron.

  • threshold – refractory dynamics of the neuron.

Returns:

spike output

Return type:

torch tensor

property v_th_step

Get voltage-threshold step parameter.

lava.lib.dl.slayer.neuron.alif.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Adaptive Resonate and Fire (Ad R&F) Neuron

Adaptive RF Izhikevich neuron.

class lava.lib.dl.slayer.neuron.adrf.Neuron(threshold, threshold_step, period, decay, threshold_decay, refractory_decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False, log_init=True)

Bases: Neuron

This is the implementation of RF neuron.

\mathfrak{Re}(z[t]) &= (1-\alpha)(\cos\phi\ \mathfrak{Re}(z[t-1]) - \sin\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Re}(x[t]) + \text{real bias}\ \mathfrak{Im}(z[t]) &= (1-\alpha)(\sin\phi\ \mathfrak{Re}(z[t-1]) + \cos\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Im}(x[t]) + \text{imag bias} \ \vartheta[t] &= (1-\alpha_{\vartheta})\, (\vartheta[t-1] - \vartheta_0) + \vartheta_0 \ r[t] &= (1-\alpha_r)\,r[t-1] \ s[t] &= |z[t]| \geq (\vartheta[t] + r[t]) \text{ and } \arg(z[t])=0

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • threshold_step (float) – the increase in threshold after spike.

  • period (float or tuple) – period of the neuron. If shared_param is False, then it can be specified as a tuple (min_period, max_period).

  • decay (float or tuple) – decay factor of the neuron. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • threshold_decay (float or tuple) – the fraction of threshold decay per time step. If shared_param is False, then it can be specified as a tuple : min_decay, max_decay).

  • refractory_decay (float or tuple) – the fraction of refractory decay per time step. If shared_param is False, then it can be specified as a tuple : min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

  • log_init (bool, optional) – if True, initialized the natural frequency in log spaced range. Default is True.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_cos_decay

The compartment cos decay parameter to be used for configuration.

property cx_refractory_decay

The compartment refractory decay parameter to be used for configuring Loihi hardware.

property cx_sin_decay

The compartment sin decay parameter to be used for configuration.

property cx_threshold_decay

The compartment threshold decay parameter to be used for configuring Loihi hardware.

property decay

The decay parameter of the neuron.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically. It is essentially a resonator dynamics with adaptive threshold and refractory response.

Parameters:

input (tuple of torch tensors) – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

  • torch tensor – real response of the neuron.

  • torch tensor – imaginary response of the neuron.

  • torch tensor – adaptive threshold of the neuron.

  • torch tensor – refractory response of the neuorn.

forward(input)

omputes the full response of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

spike response of the neuron.

Return type:

torch tensor

property frequency

The frequency of neuron oscillation.

property lam

The lambda parameter of the neuron.

property period

The period of the neuron oscillation.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(real, imag, threshold, refractory)

Extracts spike points from the real and imaginary states.

Parameters:

  • real (torch tensor) – real dynamics of the neuron.

  • imag (torch tensor) – imaginary dynamics of the neuron.

  • threshold (torch tensor) – threshold dynamics of the neuron.

  • refractory (torch tensor) – refractory dynamics of the neuron.

Returns:

spike output

Return type:

torch tensor

property v_th_step

Get voltage-threshold step parameter.

lava.lib.dl.slayer.neuron.adrf.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Adaptive Resonate and Fire Izhikevich (Ad R&F Iz) Neuron

Adaptive RF Izhikevich neuron.

class lava.lib.dl.slayer.neuron.adrf_iz.Neuron(threshold, threshold_step, period, decay, threshold_decay, refractory_decay, tau_grad=1, scale_grad=1, scale=64, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False, graded_spike=False, log_init=True)

Bases: Neuron

This is the implementation of Adaptive RF Izhikevich neuron.

\mathfrak{Re}(z[t]) &= (1-\alpha)(\cos\phi\ \mathfrak{Re}(z[t-1]) - \sin\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Re}(x[t]) + \text{real bias}\ \mathfrak{Im}(z[t]) &= (1-\alpha)(\sin\phi\ \mathfrak{Re}(z[t-1]) + \cos\phi\ \mathfrak{Im}(z[t-1])) + \mathfrak{Im}(x[t]) + \text{imag bias}\ \vartheta[t] &= (1-\alpha_{\vartheta})\, (\vartheta[t-1] - \vartheta_0) + \vartheta_0 \ r[t] &= (1-\alpha_r)\,r[t-1] \ s[t] &= \mathfrak{Im}(z[t]) \geq (\vartheta[t] + r[t])

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • threshold_step (float) – the increase in threshold after spike.

  • period (float or tuple) – period of the neuron. If shared_param is False, then it can be specified as a tuple (min_period, max_period).

  • decay (float or tuple) – decay factor of the neuron. If shared_param is False, then it can be specified as a tuple (min_decay, max_decay).

  • threshold_decay (float or tuple) – the fraction of threshold decay per time step. If shared_param is False, then it can be specified as a tuple : min_decay, max_decay).

  • refractory_decay (float or tuple) – the fraction of refractory decay per time step. If shared_param is False, then it can be specified as a tuple : min_decay, max_decay).

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

  • graded_spike (bool, optional) – flag to enable/disable graded spike output. Defaults to False.

  • log_init (bool, optional) – if True, initialized the natural frequency in log spaced range. Default is True.

clamp()

A function to clamp the sin decay and cosine decay parameters to be within valid range. The user will generally not need to call this function.

property cx_cos_decay

The compartment cos decay parameter to be used for configuration.

property cx_refractory_decay

The compartment refractory decay parameter to be used for configuring Loihi hardware.

property cx_sin_decay

The compartment sin decay parameter to be used for configuration.

property cx_threshold_decay

The compartment threshold decay parameter to be used for configuring Loihi hardware.

property decay

The decay parameter of the neuron.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

dynamics(input)

Computes the dynamics (without spiking behavior) of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically. It is essentially a resonator dynamics with Izhikevich firing with adaptive threshold and refractory response.

Parameters:

input (tuple of torch tensors) – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

  • torch tensor – real response of the neuron.

  • torch tensor – imaginary response of the neuron.

  • torch tensor – adaptive threshold of the neuron.

  • torch tensor – refractory response of the neuorn.

forward(input)

Computes the full response of the neuron instance to a complex input tuple. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (tuple of torch tensors) – Complex input tuple of tensor, i.e. (real_input, imag_input).

Returns:

spike response of the neuron.

Return type:

torch tensor

property frequency

The frequency of neuron oscillation.

property lam

The lambda parameter of the neuron.

property period

The period of the neuron oscillation.

property scale

Scale difference between slayer representation and hardware representation of the variable states.

spike(real, imag, threshold, refractory)

Extracts spike points from the real and imaginary states.

Parameters:

  • real (torch tensor) – real dynamics of the neuron.

  • imag (torch tensor) – imaginary dynamics of the neuron.

  • threshold (torch tensor) – threshold dynamics of the neuron.

  • refractory (torch tensor) – refractory dynamics of the neuron.

Returns:

spike output

Return type:

torch tensor

property v_th_step

Get voltage-threshold step parameter.

lava.lib.dl.slayer.neuron.adrf_iz.neuron_params(device_params, scale=64, p_scale=4096)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

  • p_scale (int) – parameter scale value. Default value = 1 << 12

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Sigma Delta Neuron (SDN)

Sigma Delta neuron.

class lava.lib.dl.slayer.neuron.sigma_delta.Neuron(threshold, activation, tau_grad=1, scale_grad=1, scale=64, cum_error=False, norm=None, dropout=None, shared_param=True, persistent_state=False, requires_grad=False)

Bases: Neuron

This is the implementation of Sigma-Delta wrapper neuron.

The internal state representations are scaled down compared to the actual hardware implementation. This allows for a natural range of synaptic weight values as well as the gradient parameters.

The neuron parameters like threshold, decays are represented as real values. They internally get converted to fixed precision representation of the hardware. It also provides properties to access the neuron parameters in fixed precision states. The parameters are internally clamped to the valid range.

Parameters:

  • threshold (float) – neuron threshold.

  • activation (fx-ptr or lambda) – The neuron activation class instance that needs to be wrapped by sigma-delta unit. For e.g. torch.nn.functional.relu would give sigma-delta-relu unit.

  • tau_grad (float, optional) – time constant of spike function derivative. Defaults to 1.

  • scale_grad (float, optional) – scale of spike function derivative. Defaults to 1.

  • scale (int, optional) – scale of the internal state. scale=1 will result in values in the range expected from the of Loihi hardware. Defaults to 1 << 6.

  • cum_error (bool, optional) – flag to enable/disable residual state of delta unit. Defaults to False.

  • norm (fx-ptr or lambda, optional) – normalization function on the dendrite output. None means no normalization. Defaults to None.

  • dropout (fx-ptr or lambda, optional) – neuron dropout method. None means no normalization. Defaults to None.

  • shared_param (bool, optional) – flag to enable/disable shared parameter neuron group. If it is False, individual parameters are assigned on a per-channel basis. Defaults to True.

  • persistent_state (bool, optional) – flag to enable/disable persistent state between iterations. Defaults to False.

  • requires_grad (bool, optional) – flag to enable/disable learning on neuron parameter. Defaults to False.

property device

The device memory (cpu/cuda) where the object lives.

property device_params

Dictionary of device parameters.

forward(input)

Computes the full response of the neuron instance to an input. The input shape must match with the neuron shape. For the first time, the neuron shape is determined from the input automatically.

Parameters:

input (torch tensor) – Input tensor.

Returns:

graded spike response of the neuron.

Return type:

torch tensor

property scale

Scale difference between slayer representation and hardware representation of the variable states.

set_bias(bias)

Sets the bias for sigma-delta unit

Parameters:

bias (torch tensor) – bias corresponding to each neuron.

property threshold

Neuron threshold

lava.lib.dl.slayer.neuron.sigma_delta.neuron_params(device_params, scale=64)

Translates device parameters to neuron parameters.

Parameters:

  • device_params (dictionary) – dictionary of device parameter specification.

  • scale (int) – neuron scale value. Default value = 1 << 6.

Returns:

dictionary of neuron parameters that can be used to initialize neuron class.

Return type:

dictionary

Neuron Dropout

Neuron Dropout.

class lava.lib.dl.slayer.neuron.dropout.Dropout(p=0.5, inplace=False)

Bases: Dropout3d

Neuron dropout method. It behaves similar to torch.nn.Dropout. However, dropout over time dimension is preserved, i.e. if a neuron is dropped, it remains dropped for the entire time duration.

Parameters:

  • p (float) – dropout probability.

  • inplace (bool) – inplace operation flag. Default is False.

Examples

>>> drop = Dropout(0.2, inplace=True)
>>> output = drop(input)
forward(input)

Neuron Normalization

Neuron normalization methods.

class lava.lib.dl.slayer.neuron.norm.MeanOnlyBatchNorm(num_features=None, momentum=0.1, pre_hook_fx=None)

Bases: Module

Implements mean only batch norm with optional user defined quantization using pre-hook-function. The mean of batchnorm translates to negative bias of the neuron.

Parameters:

  • num_features (int) – number of features. It is automatically initialized on first run if the value is None. Default is None.

  • momentum (float) – momentum of mean calculation. Defaults to 0.1.

  • pre_hook_fx (function pointer or lambda) – pre-hook-function that is applied to the normalization output. User can provide a quantization method as needed. Defaults to None.

num_features
momentum
pre_hook_fx
running_mean

running mean estimate.

Type:

torch tensor

update

enable mean estimte update.

Type:

bool

property bias

Equivalent bias shift.

forward(inp)
reset_parameters()

Reset states.

class lava.lib.dl.slayer.neuron.norm.WgtScaleBatchNorm(num_features=None, momentum=0.1, weight_exp_bits=3, eps=1e-05, pre_hook_fx=None)

Bases: Module

Implements batch norm with variance scale in powers of 2. This allows eventual normalizaton to be implemented with bit-shift in a hardware friendly manner. Optional user defined quantization can be enabled using a pre-hook-function. The mean of batchnorm translates to negative bias of the neuron.

Parameters:

  • num_features (int) – number of features. It is automatically initialized on first run if the value is None. Default is None.

  • momentum (float) – momentum of mean calculation. Defaults to 0.1.

  • weight_exp_bits (int) – number of allowable bits for weight exponentation. Defaults to 3.

  • eps (float) – infitesimal value. Defaults to 1e-5.

  • pre_hook_fx (function pointer or lambda) – pre-hook-function that is applied to the normalization output. User can provide a quantization method as needed. Defaults to None.

num_features
momentum
weight_exp_bits
eps
pre_hook_fx
running_mean

running mean estimate.

Type:

torch tensor

running_var

running variance estimate.

Type:

torch tensor

update

enable mean estimte update.

Type:

bool

property bias

Equivalent bias shift.

forward(inp)
reset_parameters()

Reset states.

std(var)
property weight_exp

Equivalent weight exponent value.

Module contents

class lava.lib.dl.slayer.neuron.Dropout(p=0.5, inplace=False)

Bases: Dropout3d

Neuron dropout method. It behaves similar to torch.nn.Dropout. However, dropout over time dimension is preserved, i.e. if a neuron is dropped, it remains dropped for the entire time duration.

Parameters:

  • p (float) – dropout probability.

  • inplace (bool) – inplace operation flag. Default is False.

Examples

>>> drop = Dropout(0.2, inplace=True)
>>> output = drop(input)
forward(input)