PatchEmbedding

class keras_mml.layers.core.PatchEmbedding

Turns image patches into vectors of fixed size.

The image patches should have come from the Patches layer.

This layer could optionally include position information in the embeddings by enabling the with_positions attribute.

num_patches

Number of patches in each image.

embedding_dim

Embedding dimension.

use_mml

Whether to use a matmul-less projection to embed the patches.

with_positions

Whether to include position information in the embeddings.

__init__(num_patches, embedding_dim, use_mml=True, with_positions=False, **kwargs)

Initializes a new instance of the layer.

Parameters:

• num_patches (int) – Number of patches in each image.

• embedding_dim (int) – Embedding dimension.

• use_mml (bool, default: True) – Whether to use a matmul-less projection to embed the patches.

• with_positions (bool, default: False) – Whether to include position information in the embeddings.

• **kwargs – Keyword arguments for keras.Layer.

Raises:

• ValueError – If the number of patches is not a positive integer.

• ValueError – If the embedding dimension is not a positive integer.

build(input_shape)

Build the layer.

Parameters:

input_shape (Tuple[int, int, int]) – Shape of the input.

call(inputs)

Calling method of the layer.

Parameters:

inputs (Float[ndarray, 'batch_size patch_count patch_dim']) – Inputs into the layer.

Returns:

Float[ndarray, 'batch_size patch_count embedding_dim'] – Transformed inputs.