Utility Modules

Assistant

Assistant utility for automatically load network from network description.

class lava.lib.dl.slayer.utils.assistant.Assistant(net, error, optimizer, stats=None, classifier=None, count_log=False, lam=None)

Bases: object

Assistant that bundles training, validation and testing workflow.

Parameters:

  • net (torch.nn.Module) – network to train.

  • error (object or lambda) – an error object or a lambda function that evaluates error. It is expected to take (output, target) | (output, label) as it’s argument and return a scalar value.

  • optimizer (torch optimizer) – the learning optimizer.

  • stats (slayer.utils.stats) – learning stats logger. If None, stats will not be logged. Defaults to None.

  • classifier (slayer.classifier or lambda) – classifier object or lambda function that takes output and returns the network prediction. None means regression mode. Classification steps are bypassed. Defaults to None.

  • count_log (bool) – flag to enable count log. Defaults to False.

  • lam (float) – lagrangian to merge network layer based loss. None means no such additional loss. If not None, net is expected to return the accumulated loss as second argument. It is intended to be used with layer wise sparsity loss. Defaults to None.

net
error
optimizer
stats
classifier
count_log
lam
device

the main device memory where network is placed. It is not at start and gets initialized on the first call.

Type:

torch.device or None

reduce_lr(factor=3.3333333333333335)

Reduces the learning rate of the optimizer by factor.

Parameters:

factor (float) – learning rate reduction factor. Defaults to 10/3.

test(input, target)

Testing assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

train(input, target)

Training assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

valid(input, target)

Validation assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

Filter

Time dimension filtering utilities.

class lava.lib.dl.slayer.utils.filter.FIR(fir_response=None, time_constant=1, length=20, sampling_time=1)

Bases: Module

Finite impulse response filter. The filters are not learnable. For learnable filter, use FIRBank with one filter.

Parameters:

  • fir_response (array) – Desired FIR response. If it is None, an exponentially decaying filter is initialized. Defaults to None.

  • time_constant (float) – time constant of exponentially decaying filter. Defaults to 1.

  • length (int) – length of the FIR filter to initialize. Defaults to 20.

  • sampling_time (float) – sampling time of FIR filter. Defaults to 1.

filter

impulse response of FIR filter.

Type:

torch tensor

sampling_time

sampling time of FIR filter.

Type:

float

forward(input)
class lava.lib.dl.slayer.utils.filter.FIRBank(num_filter, filter_length, sampling_time=1, scale=1)

Bases: Conv3d

Finite impulse response filter bank. The filters are learnable.

Parameters:

  • num_filter (int) – number of FIR filters in the bank.

  • filter_length (float) – time length of the filter.

  • sampling_time (float) – sampling time of the filter. Defaults to 1.

  • scale (float) – initialization scaling factor for filter. Defaults to 1.

sampling_time

sampling time of the filter. Defaults to 1.

Type:

float

property filter_length

Time length of the filter.

forward(input)
property impulse_response

Impulse response of filter bank

property num_filter

Number of filters in the bank.

lava.lib.dl.slayer.utils.filter.conv(input, filter, sampling_time=1)

Convolution in time.

Parameters:

  • input (torch tensor) – input signal. Last dimension is assumed to be time.

  • filter (torch tensor) – convolution filter. Assumed to be 1 dimensional. It will be flattened otherwise.

  • sampling_time (float) – sampling time. Defaults to 1.

Returns:

convolution output. Output shape is same as input.

Return type:

torch tensor

Examples

>>> output = conv(input, filter)
lava.lib.dl.slayer.utils.filter.corr(input, filter, sampling_time=1)

Correlation in time.

Parameters:

  • input (torch tensor) – input signal. Last dimension is assumed to be time.

  • filter (torch tensor) – correlation filter. Assumed to be 1 dimensional. It will be flattened otherwise.

  • sampling_time (float) – sampling time. Defaults to 1.

Returns:

correlation output. Output shape is same as input.

Return type:

torch tensor

Examples

>>> output = corr(input, filter)

Integer Utilities

Integer bit shift utilities.

class lava.lib.dl.slayer.utils.int_utils.Q2Zero(*args, **kwargs)

Bases: Function

Autograd compliant version of quantization towards zero.

static backward(ctx, grad_output)
static forward(ctx, x)
lava.lib.dl.slayer.utils.int_utils.right_shift_to_zero(x, bits)

Right shift with quantization towards zero implementation.

Parameters:

  • x (torch.int32 or torch.int64) – input tensor.

  • bits (int) – number of bits to shift.

Returns:

right shift to zero result.

Return type:

torch.int32 or torch.int64

Quantization

Quantization utility.

class lava.lib.dl.slayer.utils.quantize.MODE(value)

Bases: IntEnum

Quantization mode constants. Options are {ROUND : 0, FLOOR : 1}.

FLOOR = 1
ROUND = 0
lava.lib.dl.slayer.utils.quantize.quantize(input, step=1, mode=MODE.ROUND)

Implements quantization of parameters. Round or floor behavior can be selected using mode argument.

Parameters:

  • input (torch tensor) – input tensor

  • step (float) – quantization step. Default is 1.

  • mode (MODE) – quantization mode. Default is MODE.ROUND.

Returns:

quantized tensor

Return type:

torch tensor

Examples

>>> # Quantize in step of 0.5
>>> x_quantized = quantize(x, step=0.5)
lava.lib.dl.slayer.utils.quantize.quantize_hook_fx(x, scale=64, num_bits=8, descale=False)

Quantize prehook function to use in slayer synapse pre-hook for quantization.

Parameters:

  • x (torch.tensor) – Input tensor.

  • scale (int, optional) – Quantization decimal scale corresponding to 1.0 value, by default (1 << 6).

  • num_bits (int, optional) – Number of bits to use in quantization, by default 8.

  • descale (bool, optional) – Flag to descale the fixed point number to integer or keep it as fixed point number. By default False.

Returns:

Quantized tensor.

Return type:

torch.tensor

Learning Statistics

Module for managing, visualizing, and displaying learning statistics.

class lava.lib.dl.slayer.utils.stats.LearningStat

Bases: object

Learning stat manager

loss_sum

accumulated loss sum.

Type:

float

correct_samples

accumulated correct samples.

Type:

int

num_samples

number of samples accumulated.

Type:

int

min_loss

best loss recorded.

Type:

float

max_accuracy

best accuracy recorded.

Type:

float

loss_log

log of loss stats.

Type:

list of float

accuracy_log

log of accuracy stats.

Type:

list of float

best_loss

does current epoch have best loss? It is updated only after stats.update().

Type:

bool

best_accuracy

does current epoch have best accuracy? It is updated only after stats.update().

Type:

bool

__str__()

String method.

property accuracy

Current accuracy.

property loss

Current loss.

reset()

Reset stat.

update()

Update stat.

property valid_accuracy_log
property valid_loss_log
class lava.lib.dl.slayer.utils.stats.LearningStats(loss_str='loss', loss_unit='', accuracy_str='accuracy', accuracy_unit='')

Bases: object

Manages training, validation and testing stats.

training

LearningStat object to manage training statistics.

Type:

LearningStat

testing

LearningStat object to manage testing statistics.

Type:

LearningStat

validation

LearningStat object to manage validation statistics.

Type:

LearningStat

__str__()

String summary of stats.

load(path='')
new_line()

Forces stats printout on new line.

plot(figures=(1, 2), figsize=None, path=None)

Plots the training curves.

Parameters:

  • figures (tuple of ints) – figures to plot loss and accuracy. Defaults to (1, 2).

  • figsize (tuple of ints or None) – custom width and height of the figure. None means default size. Defaults to None.

  • path (str) – If not None, saves the plot to the path specified. Defaults to None.

print(epoch, iter=None, time_elapsed=None, header=None, dataloader=None)

Dynamic print method for stats.

Parameters:

  • epoch (int) – current epoch

  • iter (int or None) – iteration number in epoch. Defaults to None.

  • time_elapsed (float or None) – elapsed time. Defaults to None.

  • header (list or None) – List of strings to print before statistics. It can be used to customize additional prints. None means no header. Defaults to None.

  • dataloader (torch.dataloader or None) – torch dataloader. If not None, shows progress in the epoch. Defaults to None.

save(path='')

Saves learning stats to file

Parameters:

path (str) – Folder path to save the stats. Defaults to ‘’.

update()

Update all the stats. Typically called at the end of epoch.

Time

Tensor time manipulation utilities.

lava.lib.dl.slayer.utils.time.replicate(input, num_steps)

Replicates input in time dimension. Additional dimension of time is added at the end.

Parameters:

  • input (torch tensor) – torch input tensor.

  • num_steps (int) – number of steps to replicate.

Returns:

input replicated num_steps times in time

Return type:

torch tensor

Examples

>>> input = torch.rand(2, 3, 4)
>>> out = replicate(input, 10)
lava.lib.dl.slayer.utils.time.shift(input, shift_val, sampling_time=1)

Implements shift in time axis.

Parameters:

  • input (torch tensor) – input tensor.

  • shift_val (torch tensor or float or int) – shift tensor. If it is scalar, same shift is applied to all the spatial dimension. Otherwise, the input’s spatial dimension must match shift’s dimension.

  • sampling_time (float) – sampling time. Defaults to 1.

Returns:

shifted output

Return type:

torch tensor

Examples

>>> output = shift(input, 7)
>>> output = shift(torch.rand(1, 10, 100), torch.arange(10))

Utils

Collection of utilities.

lava.lib.dl.slayer.utils.utils.diagonal_mask(dim, num_diagonals)

Creates a binary mask with ones around the major diagonal defined by num_diagonals.

Parameters:

  • dim (int) – dimension of the mask matrix

  • num_diagonals (int) – number of diagonals. The number gets rounded up to the nearest odd number. 1 means an identity matrix.

Returns:

mask tensor

Return type:

torch tensor

class lava.lib.dl.slayer.utils.utils.dotdict

Bases: dict

Dot notation access to dictionary attributes. For e.g. my_dict["key"] is same as my_dict.key

lava.lib.dl.slayer.utils.utils.event_rate(x)

Calculate the rate of event (non-zero value) in a torch tensor. If the tensor has more than one time dimesion, first dimension is ignored as it represents initialization events.

Parameters:

x (torch.tensor) – Input torch tensor.

Returns:

Average event rate.

Return type:

float

class lava.lib.dl.slayer.utils.utils.staticproperty(fget=None, fset=None, fdel=None, doc=None)

Bases: property

wraps static member function of a class as a static property of that class.

Module contents

class lava.lib.dl.slayer.utils.Assistant(net, error, optimizer, stats=None, classifier=None, count_log=False, lam=None)

Bases: object

Assistant that bundles training, validation and testing workflow.

Parameters:

  • net (torch.nn.Module) – network to train.

  • error (object or lambda) – an error object or a lambda function that evaluates error. It is expected to take (output, target) | (output, label) as it’s argument and return a scalar value.

  • optimizer (torch optimizer) – the learning optimizer.

  • stats (slayer.utils.stats) – learning stats logger. If None, stats will not be logged. Defaults to None.

  • classifier (slayer.classifier or lambda) – classifier object or lambda function that takes output and returns the network prediction. None means regression mode. Classification steps are bypassed. Defaults to None.

  • count_log (bool) – flag to enable count log. Defaults to False.

  • lam (float) – lagrangian to merge network layer based loss. None means no such additional loss. If not None, net is expected to return the accumulated loss as second argument. It is intended to be used with layer wise sparsity loss. Defaults to None.

net
error
optimizer
stats
classifier
count_log
lam
device

the main device memory where network is placed. It is not at start and gets initialized on the first call.

Type:

torch.device or None

reduce_lr(factor=3.3333333333333335)

Reduces the learning rate of the optimizer by factor.

Parameters:

factor (float) – learning rate reduction factor. Defaults to 10/3.

test(input, target)

Testing assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

train(input, target)

Training assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

valid(input, target)

Validation assistant.

Parameters:

  • input (torch tensor) – input tensor.

  • target (torch tensor) – ground truth or label.

Returns:

  • output – network’s output.

  • count (optional) – spike count if count_log is enabled

class lava.lib.dl.slayer.utils.LearningStat

Bases: object

Learning stat manager

loss_sum

accumulated loss sum.

Type:

float

correct_samples

accumulated correct samples.

Type:

int

num_samples

number of samples accumulated.

Type:

int

min_loss

best loss recorded.

Type:

float

max_accuracy

best accuracy recorded.

Type:

float

loss_log

log of loss stats.

Type:

list of float

accuracy_log

log of accuracy stats.

Type:

list of float

best_loss

does current epoch have best loss? It is updated only after stats.update().

Type:

bool

best_accuracy

does current epoch have best accuracy? It is updated only after stats.update().

Type:

bool

__str__()

String method.

property accuracy

Current accuracy.

property loss

Current loss.

reset()

Reset stat.

update()

Update stat.

property valid_accuracy_log
property valid_loss_log
class lava.lib.dl.slayer.utils.LearningStats(loss_str='loss', loss_unit='', accuracy_str='accuracy', accuracy_unit='')

Bases: object

Manages training, validation and testing stats.

training

LearningStat object to manage training statistics.

Type:

LearningStat

testing

LearningStat object to manage testing statistics.

Type:

LearningStat

validation

LearningStat object to manage validation statistics.

Type:

LearningStat

__str__()

String summary of stats.

load(path='')
new_line()

Forces stats printout on new line.

plot(figures=(1, 2), figsize=None, path=None)

Plots the training curves.

Parameters:

  • figures (tuple of ints) – figures to plot loss and accuracy. Defaults to (1, 2).

  • figsize (tuple of ints or None) – custom width and height of the figure. None means default size. Defaults to None.

  • path (str) – If not None, saves the plot to the path specified. Defaults to None.

print(epoch, iter=None, time_elapsed=None, header=None, dataloader=None)

Dynamic print method for stats.

Parameters:

  • epoch (int) – current epoch

  • iter (int or None) – iteration number in epoch. Defaults to None.

  • time_elapsed (float or None) – elapsed time. Defaults to None.

  • header (list or None) – List of strings to print before statistics. It can be used to customize additional prints. None means no header. Defaults to None.

  • dataloader (torch.dataloader or None) – torch dataloader. If not None, shows progress in the epoch. Defaults to None.

save(path='')

Saves learning stats to file

Parameters:

path (str) – Folder path to save the stats. Defaults to ‘’.

update()

Update all the stats. Typically called at the end of epoch.

lava.lib.dl.slayer.utils.QUANTIZE_MODE

alias of MODE

lava.lib.dl.slayer.utils.diagonal_mask(dim, num_diagonals)

Creates a binary mask with ones around the major diagonal defined by num_diagonals.

Parameters:

  • dim (int) – dimension of the mask matrix

  • num_diagonals (int) – number of diagonals. The number gets rounded up to the nearest odd number. 1 means an identity matrix.

Returns:

mask tensor

Return type:

torch tensor

class lava.lib.dl.slayer.utils.dotdict

Bases: dict

Dot notation access to dictionary attributes. For e.g. my_dict["key"] is same as my_dict.key

lava.lib.dl.slayer.utils.quantize(input, step=1, mode=MODE.ROUND)

Implements quantization of parameters. Round or floor behavior can be selected using mode argument.

Parameters:

  • input (torch tensor) – input tensor

  • step (float) – quantization step. Default is 1.

  • mode (MODE) – quantization mode. Default is MODE.ROUND.

Returns:

quantized tensor

Return type:

torch tensor

Examples

>>> # Quantize in step of 0.5
>>> x_quantized = quantize(x, step=0.5)
class lava.lib.dl.slayer.utils.staticproperty(fget=None, fset=None, fdel=None, doc=None)

Bases: property

wraps static member function of a class as a static property of that class.