lava.magma.compiler

• lava.magma.compiler.builders
  • lava.magma.compiler.builders.channel_builder
  • lava.magma.compiler.builders.interfaces
  • lava.magma.compiler.builders.py_builder
  • lava.magma.compiler.builders.runtimeservice_builder
• lava.magma.compiler.channels
  • lava.magma.compiler.channels.interfaces
  • lava.magma.compiler.channels.pypychannel
• lava.magma.compiler.subcompilers
  • lava.magma.compiler.subcompilers.py
  • lava.magma.compiler.subcompilers.address
  • lava.magma.compiler.subcompilers.channel_builders_factory
  • lava.magma.compiler.subcompilers.channel_map_updater
  • lava.magma.compiler.subcompilers.constants
  • lava.magma.compiler.subcompilers.exceptions
  • lava.magma.compiler.subcompilers.interfaces

lava.magma.compiler.channel_map

class lava.magma.compiler.channel_map.ChannelMap(*args, **kwargs)

Bases: dict

The ChannelMap is used by the SubCompilers during compilation to communicate how they are planning to partition Processes onto their available resources.

classmethod from_proc_groups(proc_groups)

Initializes a ChannelMap from a list of process ProcGroups extracting the ports from every process group.

For every port pair in the process groups a PortPair will be created and set as a key on the ChannelMap object which inherits from standard dict, the values are initialized with default Payload having multiplicity of 1.

Every subcompiler will then update the multiplicity according to its partitioning process. Payload also include fields for parameterization of convolutional processes.

Parameters

proc_groups (a list of ProcessGroups each multiple processes.) –

Return type

ChannelMap

Returns

• channel_map (A ChannelMap object initialized with the PortPairs found in)

• the list of process groups given as input.

get_port_initializer(port)

has_port_initializer(port)

Return type

bool

property lmt_allocation_dict: Dict[int, int]

Return type

Dict[int, int]

set_port_initializer(port, port_initializer)

class lava.magma.compiler.channel_map.Payload(multiplicity, tiling=None, src_port_initializer=None, dst_port_initializer=None)

Bases: object

dst_port_initializer: PortInitializer = None

multiplicity: int

src_port_initializer: PortInitializer = None

tiling: Optional[Tuple[int, ...]] = None

class lava.magma.compiler.channel_map.PortPair(src, dst)

Bases: object

dst: AbstractDstPort

src: AbstractSrcPort

lava.magma.compiler.compiler

class lava.magma.compiler.compiler.Compiler(compile_config=None, loglevel=30)

Bases: object

Lava processes Compiler, called from any process in a process network.

Creates an Executable for the network of processes connected to the process passed to the compile method.

compile(process, run_cfg)

Compiles all Processes connected to the given Process and the channels defined by their connectivity.

Returns an Executable that contains all Builder instances required to execute the Lava Process network on heterogeneous hardware.

Parameters

• process (AbstractProcess) – Process from which all connected Processes in the Lava Process network are searched.

• run_cfg (RunConfig) – RunConfig that determines which ProcessModels will be selected for Processes.

Returns

executable – An instance of an Executable that contains all required Builders.

Return type

Executable

lava.magma.compiler.compiler_graphs

class lava.magma.compiler.compiler_graphs.AbstractProcGroupDiGraphs

Bases: ABC

Abstract class for generating and holding directed graphs needed to create ProcGroups for compilation.

Concrete subclass must implement get_proc_groups() method that outputs a list of ProcGroups, where type(ProcGroup) = List[AbstractProcess].

Downstream compiler iterates over the list of ProcGroups and invokes appropriate sub-compilers depending on the ProcessModel type.

abstract get_proc_groups()

Return type

List[List[AbstractProcess]]

class lava.magma.compiler.compiler_graphs.DiGraphBase(*args, **kwargs)

Bases: DiGraph

Base class for directed graphs in the compiler.

The nodes of instances of this class can be any hashable objects, they need not be of type AbstractProcess. Inherits from NetworkX.DiGraph class.

annotate_digraph_by_degree()

Annotate the graph’s nodes according to their degree.

More specifically, the annotations are based on the degree deficit of a node, defined as (out degree - in degree).

See also

NodeAnnotation

collapse_cycles_to_nodes()

Find simple cycles and collapse them on a single node iteratively, until no simple cycles can be found or the entire graph reduces to a single simple cycle.

Returns

out_graph – A copy of self with node-replacement surgery complete.

Return type

DiGraphBase

collapse_subgraph_to_node(subgraph)

Replace any connected subgraph of the DiGraphBase object with a single node, while preserving connectivity.

The new node is a DiGraphBase object, which preserves the internal connectivity of the subgraph.

Parameters

subgraph (DiGraphBase) – Subgraph which needs to be collapsed into a single node. It needs to be connected..

Returns

out_graph – A copy of self with node-replacement surgery complete.

Return type

DiGraphBase

is_dag(graph=None)

Check if the input DiGraphBase is a DAG by recursive leaf pruning.

Parameters

graph (DiGraphBase) – This parameter is only needed for recursion. It holds the residual graph after pruning leaf nodes of the graph from previous recursive step. At the first iterations, graph = None.

Return type

Tuple[bool, DiGraphBase]

Returns

• DAGness of the graph (Bool) – True if the graph is a DAG, else False

• graph (DiGraphBase) – The residual subgraph that remains after the input graph runs out of leaf nodes after all recursions are over.

class lava.magma.compiler.compiler_graphs.NodeAnnotation(value)

Bases: Enum

Annotations for the nodes of directed graphs created for compilation. The node attribute called “degdef” (short for “degree deficit”) of a node takes one of the following seven values:

1. PUREIN, a pure input Process: in degree of the node = 0

2. PUREOUT a pure output Process: out degree of the node = 0

3. ISOLATED a Process node with in degree = out degree = 0

4. INLIKE an in-like Process: (out degree) - (in degree) < 0

5. OUTLIKE an out-like Process: (out degree) - (in degree) > 0

6. NEUTRAL a Process node with in degree = out degree

7. INVALID a Process node that does not fit any of the above (unlikely)

INLIKE = 4

INVALID = 7

ISOLATED = 3

NEUTRAL = 6

OUTLIKE = 5

PUREIN = 1

PUREOUT = 2

class lava.magma.compiler.compiler_graphs.ProcDiGraph(*args, **kwargs)

Bases: DiGraphBase

Directed graph data structure for the compiler which has some nodes that are Lava Processes.

If a list of Lava Processes is passed to the constructor, using proc_graph keyword argument, a DiGraph will be created with the Processes as nodes and their interconnectivity as edges.

Inherits from DiGraphBase, which in turn inherits from NetworkX.DiGraph. If the constructor is passed *args and **kwargs that create parts of a NetworkX.DiGraph, then the graph with Process nodes is created in addition to (and disjoint from) the existing graph elements.

convert_to_procid_graph()

Convert ProcDiGraph to a DiGraph whose nodes are just the process ids of Processes from the original ProcDiGraph.

This utility method is useful to compare two ProcDiGraphs without actually using entire AbstractProcess objects as nodes.

Returns

procid_graph – A graph with nodes = process ids of Processes forming nodes of ProcDiGraph. Each node retains the attributes of the original nodes.

Return type

DiGraphBase

class lava.magma.compiler.compiler_graphs.ProcGroupDiGraphs(proc, run_cfg)

Bases: AbstractProcGroupDiGraphs

Concrete subclass of AbstractProcGroupDiGraphs that generates and holds various directed graphs needed to generate ProcGroups needed for compilation.

All sub-compilers need the following tasks to be complete before they are instantiated:

1. Find Processes:

   Using the “base” Process, on which proc.run(…) was called, a list of all processes connected to the base process is generated by traversing all InPort and OutPort connectivity.

2. Generate RawProcDiGraph:

   Using the list in step 1, a directed graph is generated with Processes as nodes. As the exact behaviour of Processes is not known (i.e., ProcessModels), the directed graph is ‘raw’, without resolving any hierarchical Processes.

3. Find and select ProcessModels:

   Using a RunConfig along with the list of Processes in step 1, a dictionary mapping Process to ProcessModel is created. Then ProcessModels are assigned as attributes to the Process instances.

4. Generate ResolvedProcDiGraph:

   Using the dict mapping Processes to ProcessModels, create a directed graph with Processes as nodes. By this point, hierarchical Processes are known and resolved using their SubProcessModels (therefore the name). As these Processes do not feature in the dict mapping, they do not constitute a node in the graph.

5. Generate IsoModelCondensedDiGraph:

   Using the ResolvedProcDiGraph generated in step 4, find all Process nodes with the same ProcessModel type, which are neighbours of each other (in the sense of Graph Theory), and collapse/condense them into a single node, while preserving the connectivity.

6. Generate ProcGroupDiGraph:

   Using the IsoModelCondensedDiGraph from step 5, find all node that are not connected via feed-forward topology (i.e., all nodes that are a part of some cyclic structure) and collapse/condense them into a single node, while preserving connectivity. Each node of ProcGroupDiGraph thus generated is a ProcGroup.

The ProcGroupDiGraph, by construction, is a single simple cycle or a DAG. In the first case, the simple cycle is broken at an arbitrary edge and List[ProcGroup] is generated from the list of its nodes. In the case that ProcGroupDiGraph is a DAG, it is ordered using topological sorting and List[ProcGroup] is produced by reversing the sorted order of nodes.

property base_proc

The Process on which Process.run(…) is called.

get_proc_groups()

Create a list of process groups sorted in compilation order.

The compilation order is obtained by reversing the topologically sorted order on compile_group_graph. Nodes of compile_group_graph can be graphs themselves. These are flattened into list of processes comprising a ProcGroup.

This is the interface to the downstream parts of the compiler.

Returns

proc_groups – A list of ProcGroup. The object ProcGroup is itself a short-hand for List[AbstractProcess]. Each ProcGroup is processed as a single entity by the downstream parts of the compiler.

Return type

List[ProcGroup]

property isomodel_condensed_digraph

Directed graph derived from ProcGroupDiGraphs.resolved_proc_graph, such that all connected Processes with same type of ProcessModel are collapsed/condensed onto a single node in the graph.

property num_procs_post_subproc_expansion

Number of leaf Processes, after ProcessModel discovery and SubProcessModel resolution/expansion.

This is the number of nodes in resolved_proc_digraph

See also

resolved_proc_digraph

property num_procs_pre_pm_discovery

Number of Processes before ProcessModel discovery.

This is the number of nodes in raw_proc_digraph.

See also

raw_proc_digraph

property proc_group_digraph

Directed graph derived from ProcGroupDiGraphs.isomodel_condensed_graph, such that all nodes involved in any kind of non-linear/feedback/cyclical topology are collapsed/condensed onto a single node.

A proc_group_digraph - by construction - is a single simple cycle or a DAG.

property raw_proc_digraph

Directed graph of all connected Processes _before_ ProcessModel discovery.

This graph is generated using Process-level connectivity before any ProcessModels are discovered. Any sub-Processes inside a SubProcessModel of a hierarchical Process will not show up in this graph.

property resolved_proc_digraph

Directed graph of all connected Processes _after_ ProcessModel discovery.

This graph is generated after discovering all ProcessModels and resolving/building SubProcessModels. Therefore, no hierarchical Processes show up in this graph. They are substituted by the sub-Processes inside their SubProcessModels.

class lava.magma.compiler.compiler_graphs.ProcessModelTypes(value)

Bases: Enum

Enumeration of different types of ProcessModels: Py, C, Nc, etc.

C(proc_params) = <class 'lava.magma.core.model.c.model.AbstractCProcessModel'>

NC(proc_params, loglevel=30) = <class 'lava.magma.core.model.nc.model.AbstractNcProcessModel'>

PY(proc_params, loglevel=30) = <class 'lava.magma.core.model.py.model.AbstractPyProcessModel'>

lava.magma.compiler.compiler_graphs.find_processes(proc, seen_procs=None)

Find all processes that are connected to proc.

Processes are connected via different kinds of Ports to other processes. This method starts at the given proc and traverses the graph along connections to find all connected processes.

During compilation, this method is called before discovering ProcessModels implementing Processes.

Parameters

• proc (AbstractProcess) – Base process starting which the discovery of all connected Processes begins.

• seen_procs (List[AbstractProcess]) – A list of Processes visited during traversal of connected Processes. Used for making the method recursive. This parameter is set to None at the time of the first call.

Returns

seen_procs – A list of all discovered Processes.

Return type

List[AbstractProcess]

lava.magma.compiler.compiler_graphs.flatten_list_itertools(ll)

Simpler way to flatten nested lists.

Return type

List

lava.magma.compiler.compiler_graphs.flatten_list_recursive(ll)

Recursively flatten a list of lists.

Parameters

ll (list) – Any list of lists (of any depth)

Returns

ll – Flattened list

Return type

list

Notes

Taken from: https://stackabuse.com/python-how-to-flatten-list-of-lists/

lava.magma.compiler.compiler_utils

lava.magma.compiler.compiler_utils.split_proc_builders_by_type(proc_builders)

Given a dictionary of process to builders, returns a tuple of process to builder dictionaries for Py, C and Nc processes.

lava.magma.compiler.exceptions

exception lava.magma.compiler.exceptions.NoProcessModelFound(proc)

Bases: Exception

exception lava.magma.compiler.exceptions.ProcessAlreadyCompiled(proc)

Bases: Exception

lava.magma.compiler.executable

class lava.magma.compiler.executable.Executable(proc_builders, channel_builders, node_configs, sync_domains, runtime_service_builders=None, sync_channel_builders=None)

Bases: object

Produced by compiler and contains everything the Runtime needs to run process.

This includes all ProcessModels of sub processes, RuntimeService processes for the various nodes in the system and channel configurations. An Executable should be serializable so it can be saved and loaded at a later point.

assign_runtime_to_all_processes(runtime)

channel_builders: ty.List[ChannelBuilderMp]

node_configs: ty.List[NodeConfig]

proc_builders: ty.Dict[AbstractProcess, 'AbstractProcessBuilder']

runtime_service_builders: ty.Optional[ty.Dict[SyncDomain, RuntimeServiceBuilder]] = None

sync_channel_builders: ty.Optional[ty.Iterable[AbstractChannelBuilder]] = None

sync_domains: ty.List[SyncDomain]

lava.magma.compiler.mappable_interface

class lava.magma.compiler.mappable_interface.Mappable

Bases: ABC

Interface to make entity mappable.

get_logical()

Return type

List of LogicalAddresses.

set_physical(addr)

Parameters

addr (List of PhysicalAddresses to be assigned to the mappable.) –

lava.magma.compiler.mapper

class lava.magma.compiler.mapper.Mapper

Bases: object

Assigns virtual addresses to different processes, mappable by mapping logical addresses to virtual addresses.

map_cores(executable, channel_map)

This function gets called from the Compiler class once the partition is done. It maps logical addresses to virtual addresses.

Parameters

executable (Compiled Executable) –

Return type

None

lava.magma.compiler.node

class lava.magma.compiler.node.Node(node_type, processes)

Bases: object

A Node represents a physical compute node on which one or more processes execute.

Nodes are of a specific type and hold references to all processes mapped to a node.

add_process(process)

class lava.magma.compiler.node.NodeConfig(init_list=None)

Bases: UserList

A NodeConfig is a collection of Nodes. Nodes represent a physical compute node on which one or more processes execute.

A NodeCfg has a list of all ‘nodes’ and a ‘node_map’ that maps each process to its respective node.

append(node)

Appends a new node to the NodeConfig.

property nodes: List[Node]

Returns list of all nodes of the NodeConfig.

Return type

List[Node]

set_var_models(var_models)

lava.magma.compiler.utils

class lava.magma.compiler.utils.LoihiCInPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None, connected_port_type=None, connected_port_encoding_type=None, spike_type=None)

Bases: LoihiIOPortInitializer

embedded_core = 0

embedded_counters = None

class lava.magma.compiler.utils.LoihiConnectedPortEncodingType(value)

Bases: IntEnum

Encoding type of the connected port - Required in case of C_PY

SEQ_DENSE = 2

SEQ_SPARSE = 4

VEC_DENSE = 1

VEC_SPARSE = 3

class lava.magma.compiler.utils.LoihiConnectedPortType(value)

Bases: IntEnum

Types of port connectivity; direction does not matter

C_C = 2

C_NC = 1

C_PY = 3

class lava.magma.compiler.utils.LoihiIOPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None, connected_port_type=None, connected_port_encoding_type=None, spike_type=None)

Bases: LoihiPortInitializer

Port Initializer for a I/O Port for C/NC Models

connected_port_encoding_type: Optional[LoihiConnectedPortEncodingType] = None

connected_port_type: Optional[LoihiConnectedPortType] = None

spike_type: Optional[SpikeType] = None

class lava.magma.compiler.utils.LoihiInPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None, connected_port_type=None, connected_port_encoding_type=None, spike_type=None)

Bases: LoihiIOPortInitializer

Port Initializer for a InPort for C/NC Models

d_type: type

name: str

port_type: str

shape: Tuple[int, ...]

size: int

class lava.magma.compiler.utils.LoihiOutPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None, connected_port_type=None, connected_port_encoding_type=None, spike_type=None)

Bases: LoihiIOPortInitializer

Port Initializer for a OutPort for C/NC Models

d_type: type

name: str

port_type: str

shape: Tuple[int, ...]

size: int

class lava.magma.compiler.utils.LoihiPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None)

Bases: PortInitializer, Mappable

This address needs to be defined based on var model

get_logical()

Return type

Returns logical address of the port initializer.

set_virtual(addrs)

Sets physical address of the port initializer :type addrs: List[NcVirtualAddress] :param addrs: :type addrs: List of address

var_model: Optional[LoihiVarModel] = None

class lava.magma.compiler.utils.LoihiVarInitializer(name, shape, value, var_id, d_type)

Bases: VarInitializer

d_type: type

class lava.magma.compiler.utils.LoihiVarPortInitializer(name, shape, d_type, port_type, size, transform_funcs=None, var_model=None)

Bases: LoihiPortInitializer

d_type: type

name: str

port_type: str

shape: Tuple[int, ...]

size: int

class lava.magma.compiler.utils.PortInitializer(name, shape, d_type, port_type, size, transform_funcs=None)

Bases: object

d_type: type

name: str

port_type: str

shape: Tuple[int, ...]

size: int

transform_funcs: Dict[str, List[partial]] = None

class lava.magma.compiler.utils.VarInitializer(name, shape, value, var_id)

Bases: object

name: str

shape: Tuple[int, ...]

value: Any

var_id: int

class lava.magma.compiler.utils.VarPortInitializer(name, shape, var_name, d_type, port_type, size, port_cls, transform_funcs=None)

Bases: object

d_type: type

name: str

port_cls: type

port_type: str

shape: Tuple[int, ...]

size: int

transform_funcs: Dict[str, List[partial]] = None

var_name: str

lava.magma.compiler.var_model

class lava.magma.compiler.var_model.AbstractVarModel(var=None, node_id=-1, runtime_srv_id=-1)

Bases: ABC

node_id: int = -1

runtime_srv_id: int = -1

var: InitVar[Var] = None

class lava.magma.compiler.var_model.CVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None)

Bases: LoihiVarModel

lava.magma.compiler.var_model.ChipIdx: int

class lava.magma.compiler.var_model.ConvInVarModel(var=None, node_id=-1, runtime_srv_id=-1, x_dim=0, y_dim=0, f_dim=0, x_split=0, f_split=0, regions=None)

Bases: AbstractVarModel, Mappable

f_dim: int = 0

f_split: int = 0

get_logical()

Return type

Returns logical address of the port initializer.

regions: ty.List[Region] = None

set_virtual(addrs)

Sets physical address of the port initializer :type addrs: List[NcVirtualAddress] :param addrs: :type addrs: List of address

x_dim: int = 0

x_split: int = 0

y_dim: int = 0

class lava.magma.compiler.var_model.ConvNeuronVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None, alloc_dims=None, valid_dims=None, var_shape=None)

Bases: LoihiNeuronVarModel

alloc_dims: ty.List[ty.Tuple[int, int, int]] = None

valid_dims: ty.List[ty.Tuple[int, int, int]] = None

var_shape: ty.Tuple[int, int, int] = None

lava.magma.compiler.var_model.CoreIdx: int

class lava.magma.compiler.var_model.LoihiAddress(physical_chip_id, physical_core_id, logical_chip_id, logical_core_id, logical_idx_addr, length, stride)

Bases: object

length: int

logical_chip_id: ChipIdx

logical_core_id: CoreIdx

logical_idx_addr: int

physical_chip_id: ChipIdx

physical_core_id: CoreIdx

stride: int

class lava.magma.compiler.var_model.LoihiNeuronAddress(physical_chip_id, physical_core_id, logical_chip_id, logical_core_id, logical_idx_addr, length, stride, neuron_group_id)

Bases: LoihiAddress

neuron_group_id: int

class lava.magma.compiler.var_model.LoihiNeuronVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None)

Bases: LoihiVarModel

class lava.magma.compiler.var_model.LoihiSynapseAddress(physical_chip_id, physical_core_id, logical_chip_id, logical_core_id, logical_idx_addr, length, stride, syn_entry_id)

Bases: LoihiAddress

syn_entry_id: int

class lava.magma.compiler.var_model.LoihiSynapseVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None)

Bases: LoihiVarModel

class lava.magma.compiler.var_model.LoihiVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None)

Bases: AbstractVarModel, Mappable

address: ty.List[LoihiAddress] = None

field_name: ty.Optional[str] = None

get_logical()

Return type

Returns logical address of the port initializer.

register_length: ty.Optional[int] = None

register_name: ty.Optional[str] = None

set_virtual(addrs)

Sets physical address of the port initializer :type addrs: List[NcVirtualAddress] :param addrs: :type addrs: List of address

sub_type: ty.Optional[str] = None

union_type: ty.Optional[bool] = False

variable_length: ty.Optional[int] = None

variable_offset: ty.Optional[int] = None

class lava.magma.compiler.var_model.NcVarModel(var=None, node_id=-1, runtime_srv_id=-1, address=None, field_name=None, register_name=None, register_length=None, variable_offset=None, variable_length=None, union_type=False, sub_type=None)

Bases: LoihiVarModel

class lava.magma.compiler.var_model.PyVarModel(var=None, node_id=-1, runtime_srv_id=-1)

Bases: AbstractVarModel

class lava.magma.compiler.var_model.Region(x_min, x_max, y_min, y_max, logical_chip_idx, logical_core_idx, physical_chip_idx=None, physical_core_idx=None)

Bases: object

logical_chip_idx: ChipIdx

logical_core_idx: CoreIdx

physical_chip_idx: ChipIdx = None

physical_core_idx: CoreIdx = None

x_max: int

x_min: int

y_max: int

y_min: int