Synapse Module

Real Synapse

Synapse module

class lava.lib.dl.slayer.synapse.layer.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Convolution synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the convolution kernel.

  • stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
kernel
stride
padding
dilation
groups
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.layer.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ConvTranspose3d, GenericLayer

Transposed convolution synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.

  • stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
kernel_size
stride
padding
dilation
groups
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.layer.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Dense synapse layer.

Parameters:

  • in_neurons (int) – number of input neurons.

  • out_neurons (int) – number of output neurons.

  • weight_scale (int) – weight initialization scaling factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

in_channels
out_channels
weight
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCT or NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.layer.GenericLayer

Bases: Module

Abstract synapse layer class.

weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

disable_weight_norm()

Disables weight normalization on synapse.

enable_weight_norm()

Enables weight normalization on synapse.

property grad_norm

Norm of weight gradients. Useful for monitoring gradient flow.

property pre_hook_fx

Returns the pre-hook function for synapse operation. Typically intended to define the quantization method.

property shape

Shape of the synapse

class lava.lib.dl.slayer.synapse.layer.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Pooling synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
kernel_size
stride
padding
dilation
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.layer.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ConvTranspose3d, GenericLayer

Unpooling synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
stride
padding
dilation
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

Complex Module

Complex synapse

class lava.lib.dl.slayer.synapse.complex.ComplexLayer

Bases: Module

Abstract complex layer class.

disable_weight_norm()

Disables weight normalization on synapse.

enable_weight_norm()

Enables weight normalization on synapse.

forward(input)

Forward complex synaptic operation.

property grad_norm

Norm of weight gradients. Useful for monitoring gradient flow.

property pre_hook_fx

Returns the pre-hook function for synapse operation. Typically intended to define the quantization method.

property shape

Shape of the synapse

class lava.lib.dl.slayer.synapse.complex.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ComplexLayer

Convolution complex-synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the convolution kernel.

  • stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

real

real synapse.

Type:

slayer.synapse.Conv

imag

imaginary synapse.

Type:

slayer.synapse.Conv

complex

True. Indicates synapse is complex.

Type:

bool

class lava.lib.dl.slayer.synapse.complex.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ComplexLayer

Transposed convolution synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.

  • stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

real

real synapse.

Type:

slayer.synapse.ConvTranspose

imag

imaginary synapse.

Type:

slayer.synapse.ConvTranspose

complex

True. Indicates synapse is complex.

Type:

bool

class lava.lib.dl.slayer.synapse.complex.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ComplexLayer

Dense compelx-synapse layer.

Parameters:

  • in_neurons (int) – number of input neurons.

  • out_neurons (int) – number of output neurons.

  • weight_scale (int) – weight initialization scaling factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

real

real synapse.

Type:

slayer.synapse.Dense

imag

imaginary synapse.

Type:

slayer.synapse.Dense

complex

True. Indicates synapse is complex.

Type:

bool

class lava.lib.dl.slayer.synapse.complex.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ComplexLayer

Pooling complex-synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

real

real synapse.

Type:

slayer.synapse.Pool

imag

imaginary synapse.

Type:

slayer.synapse.Pool

complex

True. Indicates synapse is complex.

Type:

bool

class lava.lib.dl.slayer.synapse.complex.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ComplexLayer

Unpooling complex-synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

real

real synapse.

Type:

slayer.synapse.Unpool

imag

imaginary synapse.

Type:

slayer.synapse.Unpool

complex

True. Indicates synapse is complex.

Type:

bool

Module contents

class lava.lib.dl.slayer.synapse.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Convolution synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the convolution kernel.

  • stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
kernel
stride
padding
dilation
groups
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ConvTranspose3d, GenericLayer

Transposed convolution synapse layer.

Parameters:

  • in_features (int) – number of input features.

  • out_features (int) – number of output features.

  • kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.

  • stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.

  • padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.

  • groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
kernel_size
stride
padding
dilation
groups
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Dense synapse layer.

Parameters:

  • in_neurons (int) – number of input neurons.

  • out_neurons (int) – number of output neurons.

  • weight_scale (int) – weight initialization scaling factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

in_channels
out_channels
weight
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCT or NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: Conv3d, GenericLayer

Pooling synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
kernel_size
stride
padding
dilation
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor

class lava.lib.dl.slayer.synapse.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)

Bases: ConvTranspose3d, GenericLayer

Unpooling synape layer.

Parameters:

  • kernel_size (int) – [description]

  • stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.

  • padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.

  • dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.

  • weight_scale (int) – weight initialization scale factor. Defaults to 1.

  • weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.

  • pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.

Note

For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.

in_channels
out_channels
weight
stride
padding
dilation
weight_norm_enabled

flag indicating weather weight norm in enabled or not.

Type:

bool

complex

False. Indicates synapse is not complex.

Type:

bool

forward(input)

Applies the synapse to the input.

Parameters:

input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.

Returns:

dendrite accumulation / weighted spikes.

Return type:

torch tensor