Axon Module

Axon Delay

Axon delay implementation.

class lava.lib.dl.slayer.axon.delay.Delay(sampling_time=1, max_delay=None, grad_scale=1)

Bases: Module

Learnable axonal delay module. The delays operate on channel dimension.

Parameters:

sampling_time (int) – Sampling time of delay. Defaults to 1.
max_delay (int) – Maximum allowable delay. Defaults to None.
grad_scale (float) – gradient scale parameter. Defaults to 1.

sampling_time

max_delay

grad-scale

delay

the delay parameter.

Type:: torch parameter

Examples

>>> axon_delay = Delay()
>>> x_delayed = axon_delay(x)

clamp(): Clamps delay to allowable range. Typically it is not needed to be called explicitly.

forward(input)

Apply delay to input tensor.

Parameters:: input (torch.tensor) – input tensor.
Returns:: delayed tensor.
Return type:: torch.tensor

property shape: Shape of the delay.

lava.lib.dl.slayer.axon.delay.delay(input, delay_val=1, sampling_time=1)

Delay the signal in time.

Parameters:

input (torch.tensor) – Input signal. The last dimension is assumed to be time dimension.
delay (int) – Amount of delay to apply. Defaults to 1.
sampling_time (int) – Sampling time of delay operation. Defaults to 1.

Returns:

delayed signal

Return type:

torch.tensor

Examples

>>> x_delayed = delay(x, 2) # delay x by 2 timesteps

Delta Encoder

Delta encoder implementation.

class lava.lib.dl.slayer.axon.delta.Delta(threshold, scale=64, tau_grad=1, scale_grad=1, cum_error=False, shared_param=True, persistent_state=False, requires_grad=False)

Bases: Module

Implements delta differential encoding followed by thresholding. The thresholds are learnable, individually or as a group.

\Delta x[t] &= x[t] - x[t-1] + r[t-1] \\ y[t] &= \begin{cases} \Delta x[t] &\text{ if } \Delta x[t] \geq \vartheta \\ 0 &\text{ otherwise} \end{cases}\\ r[t] &= \Delta x[t] - y[t]

For cumulative error, output evaluation is changed to

e[t] &= e[t] + \Delta x[t]\\ y[t] &= \begin{cases} \Delta x[t] &\text{ if } e[t] \geq \vartheta \\ 0 &\text{ otherwise}\\ e[t] &= e[t] * (1 - \mathcal{H}(|y[t]|)) \end{cases}

Parameters:

threshold (float) – threshold value.
scale (int) – quantization step size. Defaults to 64.
tau_grad (float) – threshold gradient relaxation parameter. Defaults to 1.
scale_grad (float) – threshold gradient scaling parameter. Defaults to 1.
cum_error (bool) – flag to enable cumulative error before thresholding. Defaults to False.
shared_param (bool) – flag to enable shared threshold. Defaults to True.
persistent_state (bool) – flag to enable persistent delta states. Defaults to False.
requires_grad (bool) – flag to enable threshold gradient. Defaults to False.

scale

tau_grad

scale_grad

cum_error

shared_param

persistent_state

requires_grad

shape

shape of delta block. It is identified on runtime. The value is None before that.

Type:: torch shape

pre_state

previous state of delta unit.

Type:: torch tensor

residual_state

residual state of delta unit.

Type:: torch tensor

error_state

error state of delta unit.

Type:: torch tensor

Examples

>> delta = Delta(threshold=1) >> y = delta(x) # differential threshold encoding

clamp(): Clamps the threshold value to [\verb~1/scale~, \infty).

property device

Device property of object

Returns:: returns the device memory where the object lives.
Return type:: torch.device

forward(input)

Module contents

class lava.lib.dl.slayer.axon.Delay(sampling_time=1, max_delay=None, grad_scale=1)

Bases: Module

Learnable axonal delay module. The delays operate on channel dimension.

Parameters:

sampling_time (int) – Sampling time of delay. Defaults to 1.
max_delay (int) – Maximum allowable delay. Defaults to None.
grad_scale (float) – gradient scale parameter. Defaults to 1.

sampling_time

max_delay

grad-scale

delay

the delay parameter.

Type:: torch parameter

Examples

>>> axon_delay = Delay()
>>> x_delayed = axon_delay(x)

clamp(): Clamps delay to allowable range. Typically it is not needed to be called explicitly.

forward(input)

Apply delay to input tensor.

Parameters:: input (torch.tensor) – input tensor.
Returns:: delayed tensor.
Return type:: torch.tensor

property shape: Shape of the delay.

class lava.lib.dl.slayer.axon.Delta(threshold, scale=64, tau_grad=1, scale_grad=1, cum_error=False, shared_param=True, persistent_state=False, requires_grad=False)

Bases: Module

Implements delta differential encoding followed by thresholding. The thresholds are learnable, individually or as a group.

\Delta x[t] &= x[t] - x[t-1] + r[t-1] \\ y[t] &= \begin{cases} \Delta x[t] &\text{ if } \Delta x[t] \geq \vartheta \\ 0 &\text{ otherwise} \end{cases}\\ r[t] &= \Delta x[t] - y[t]

For cumulative error, output evaluation is changed to

e[t] &= e[t] + \Delta x[t]\\ y[t] &= \begin{cases} \Delta x[t] &\text{ if } e[t] \geq \vartheta \\ 0 &\text{ otherwise}\\ e[t] &= e[t] * (1 - \mathcal{H}(|y[t]|)) \end{cases}

Parameters:

threshold (float) – threshold value.
scale (int) – quantization step size. Defaults to 64.
tau_grad (float) – threshold gradient relaxation parameter. Defaults to 1.
scale_grad (float) – threshold gradient scaling parameter. Defaults to 1.
cum_error (bool) – flag to enable cumulative error before thresholding. Defaults to False.
shared_param (bool) – flag to enable shared threshold. Defaults to True.
persistent_state (bool) – flag to enable persistent delta states. Defaults to False.
requires_grad (bool) – flag to enable threshold gradient. Defaults to False.

scale

tau_grad

scale_grad

cum_error

shared_param

persistent_state

requires_grad

shape

shape of delta block. It is identified on runtime. The value is None before that.

Type:: torch shape

pre_state

previous state of delta unit.

Type:: torch tensor

residual_state

residual state of delta unit.

Type:: torch tensor

error_state

error state of delta unit.

Type:: torch tensor

Examples

>> delta = Delta(threshold=1) >> y = delta(x) # differential threshold encoding

clamp(): Clamps the threshold value to [\verb~1/scale~, \infty).

property device

Device property of object

Returns:: returns the device memory where the object lives.
Return type:: torch.device

forward(input)

lava.lib.dl.slayer.axon.delay(input, delay_val=1, sampling_time=1)

Delay the signal in time.

Parameters:

input (torch.tensor) – Input signal. The last dimension is assumed to be time dimension.
delay (int) – Amount of delay to apply. Defaults to 1.
sampling_time (int) – Sampling time of delay operation. Defaults to 1.

Returns:

delayed signal

Return type:

torch.tensor

Examples

>>> x_delayed = delay(x, 2) # delay x by 2 timesteps