LRUMML

class keras_mml.layers.recurrent.LRUMML

Linear Recurrent Unit (LRU) layer, mostly without matrix multiplications.

The core algorithm of this layer mostly follows the implementation in Resurrecting Recurrent Neural Networks for Long Sequences (see Appendix A), with a few modifications from the LRU-pytorch GitHub repository. We replace some matrix multiplications with ternary weights, although the option to use it as fully matrix multiplication free is available using the fully_mml attribute.

units

Dimensionality of the output space.

state_dim

Dimensionality of the internal state space.

fully_mml

Whether to use matmul-free operations for all the layers.

r_min

Minimum modulus of the complex weights in \(\mathbf{\Lambda}\).

r_max

Maximum modulus of the complex weights in \(\mathbf{\Lambda}\).

max_phase

Maximum phase of the complex weights in \(\mathbf{\Lambda}\).

use_bias

Whether to use a bias vector for the layer.

__init__(units, state_dim, fully_mml=False, r_min=0, r_max=1, max_phase=6.283185307179586, use_bias=False, **kwargs)

Initializes a new instance of the layer.

Parameters:

• units (int) – Dimensionality of the output space.

• state_dim (int) – Dimensionality of the internal state space.

• fully_mml (bool, default: False) – Whether to use matmul-free operations for all the layers.

• r_min (float, default: 0) – Minimum modulus of the complex weights in \(\mathbf{\Lambda}\).

• r_max (float, default: 1) – Maximum modulus of the complex weights in \(\mathbf{\Lambda}\).

• max_phase (float, default: 6.283185307179586) – Maximum phase of the complex weights in \(\mathbf{\Lambda}\).

• use_bias (bool, default: False) – Whether to use a bias vector for the layer.

• **kwargs – Keyword arguments for keras.Layer.

Raises:

• ValueError – If the units provided is not a positive integer.

• ValueError – If the state dimensionality provided is not a positive integer.

call(sequences, initial_state=None, mask=None, training=False)

Calling method of the layer.

Parameters:

• sequences (Float[ndarray, 'batch_size timesteps features']) – Inputs into the layer.

• initial_state (Optional[List], default: None) – List of initial state tensors to be passed to the first call of the cell. If not provided, will cause creation of zero-filled initial state tensors.

• mask (Optional[Any], default: None) – Binary tensor indicating whether a given timestep should be masked. An individual True entry indicates that the corresponding timestep should be utilized, while a False entry indicates that the corresponding timestep should be ignored.

• training (bool, default: False) – Indicates whether the layer should behave in training mode or in inference mode. This argument is passed to the cell when calling it.

Returns:

Float[ndarray, 'batch_size timesteps'] – Transformed inputs.

classmethod from_config(config)

Creates the layer from the given configuration.

Parameters:

config (Dict[str, Any]) – Configuration dictionary.

Returns:

LRUMML – Created instance.

get_config()

Gets the configuration for the layer.

Returns:

Dict[str, Any] – Layer configuration.