Synapse Module
Real Synapse
Synapse module
- class lava.lib.dl.slayer.synapse.layer.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Convolution synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the convolution kernel.
stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- kernel
- stride
- padding
- dilation
- groups
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.layer.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ConvTranspose3d
,GenericLayer
Transposed convolution synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.
stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- kernel_size
- stride
- padding
- dilation
- groups
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.layer.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Dense synapse layer.
- Parameters:
in_neurons (int) – number of input neurons.
out_neurons (int) – number of output neurons.
weight_scale (int) – weight initialization scaling factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
- in_channels
- out_channels
- weight
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCT or NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.layer.GenericLayer
Bases:
Module
Abstract synapse layer class.
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- disable_weight_norm()
Disables weight normalization on synapse.
- enable_weight_norm()
Enables weight normalization on synapse.
- property grad_norm
Norm of weight gradients. Useful for monitoring gradient flow.
- property pre_hook_fx
Returns the pre-hook function for synapse operation. Typically intended to define the quantization method.
- property shape
Shape of the synapse
- class lava.lib.dl.slayer.synapse.layer.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Pooling synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- kernel_size
- stride
- padding
- dilation
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.layer.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ConvTranspose3d
,GenericLayer
Unpooling synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- stride
- padding
- dilation
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
Complex Module
Complex synapse
- class lava.lib.dl.slayer.synapse.complex.ComplexLayer
Bases:
Module
Abstract complex layer class.
- disable_weight_norm()
Disables weight normalization on synapse.
- enable_weight_norm()
Enables weight normalization on synapse.
- forward(input)
Forward complex synaptic operation.
- property grad_norm
Norm of weight gradients. Useful for monitoring gradient flow.
- property pre_hook_fx
Returns the pre-hook function for synapse operation. Typically intended to define the quantization method.
- property shape
Shape of the synapse
- class lava.lib.dl.slayer.synapse.complex.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ComplexLayer
Convolution complex-synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the convolution kernel.
stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- real
real synapse.
- Type:
- imag
imaginary synapse.
- Type:
- complex
True. Indicates synapse is complex.
- Type:
bool
- class lava.lib.dl.slayer.synapse.complex.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ComplexLayer
Transposed convolution synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.
stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- real
real synapse.
- imag
imaginary synapse.
- complex
True. Indicates synapse is complex.
- Type:
bool
- class lava.lib.dl.slayer.synapse.complex.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ComplexLayer
Dense compelx-synapse layer.
- Parameters:
in_neurons (int) – number of input neurons.
out_neurons (int) – number of output neurons.
weight_scale (int) – weight initialization scaling factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
- real
real synapse.
- Type:
- imag
imaginary synapse.
- Type:
- complex
True. Indicates synapse is complex.
- Type:
bool
- class lava.lib.dl.slayer.synapse.complex.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ComplexLayer
Pooling complex-synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- real
real synapse.
- Type:
- imag
imaginary synapse.
- Type:
- complex
True. Indicates synapse is complex.
- Type:
bool
- class lava.lib.dl.slayer.synapse.complex.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ComplexLayer
Unpooling complex-synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- real
real synapse.
- Type:
- imag
imaginary synapse.
- Type:
- complex
True. Indicates synapse is complex.
- Type:
bool
Module contents
- class lava.lib.dl.slayer.synapse.Conv(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Convolution synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the convolution kernel.
stride (int or tuple of two ints) – stride of the convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- kernel
- stride
- padding
- dilation
- groups
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.ConvTranspose(in_features, out_features, kernel_size, stride=1, padding=0, dilation=1, groups=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ConvTranspose3d
,GenericLayer
Transposed convolution synapse layer.
- Parameters:
in_features (int) – number of input features.
out_features (int) – number of output features.
kernel_size (int or tuple of two ints) – size of the transposed convolution kernel.
stride (int or tuple of two ints) – stride of the transposed convolution. Defaults to 1.
padding (int or tuple of two ints) – padding of the transposed convolution. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the transposed convolution. Defaults to 1.
groups (int) – number of blocked connections from input channel to output channel. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- kernel_size
- stride
- padding
- dilation
- groups
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.Dense(in_neurons, out_neurons, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Dense synapse layer.
- Parameters:
in_neurons (int) – number of input neurons.
out_neurons (int) – number of output neurons.
weight_scale (int) – weight initialization scaling factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
- in_channels
- out_channels
- weight
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCT or NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.Pool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
Conv3d
,GenericLayer
Pooling synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of pooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the pooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the pooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- kernel_size
- stride
- padding
- dilation
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor
- class lava.lib.dl.slayer.synapse.Unpool(kernel_size, stride=None, padding=0, dilation=1, weight_scale=1, weight_norm=False, pre_hook_fx=None)
Bases:
ConvTranspose3d
,GenericLayer
Unpooling synape layer.
- Parameters:
kernel_size (int) – [description]
stride (int or tuple of two ints) – stride of unpooling. Defaults to kernel_size.
padding (int or tuple of two ints) – padding of the unpooling. Defaults to 0.
dilation (int or tuple of two ints) – dilation of the unpooling. Defaults to 1.
weight_scale (int) – weight initialization scale factor. Defaults to 1.
weight_norm (bool) – flag to enable/disable weight normalization. Defaults to False.
pre_hook_fx (optional) – a function reference or a lambda function. If the function is provided, it will be applied to it’s weight before the forward operation of the synapse. Typically the function is a quantization mechanism of the synapse. Defaults to None.
Note
For kernel_size, stride, padding and dilation, the tuple of two ints are represented in (height, width) order. The integer value is broadcast to height and width.
- in_channels
- out_channels
- weight
- stride
- padding
- dilation
- weight_norm_enabled
flag indicating weather weight norm in enabled or not.
- Type:
bool
- complex
False. Indicates synapse is not complex.
- Type:
bool
- forward(input)
Applies the synapse to the input.
- Parameters:
input (torch tensor) – Input tensor. Typically spikes. Input is expected to be of shape NCHWT.
- Returns:
dendrite accumulation / weighted spikes.
- Return type:
torch tensor