keras_mml.layers.activationsΒΆ

Activation layers implemented by Keras-MML.

Classes

GLUMML

General Gated Linear Unit (GLU) without matrix multiplications.

BilinearMML

Gated Linear Unit (GLU) without matrix multiplications and any activation functions.

GeGLUMML

Gaussian Error Linear Unit (GELU) activated Gated Linear Unit (GLU) without matrix multiplications.

ReGLUMML

Rectified Linear Unit (ReLU) activated Gated Linear Unit (GLU) without matrix multiplications.

SeGLUMML

Scaled Exponential Linear Unit (SELU) activated Gated Linear Unit (GLU) without matrix multiplications.

SwiGLUMML

Swish with \(\beta = 1\) (i.e., Sigmoid Linear Unit (SiLU)) activated Gated Linear Unit (GLU) without matrix multiplications.