Welcome to Keras-MatMulLess (Keras-MML)’s documentation!¶
We offer no explanation as to why these architectures seem to work; we attribute their success, as all else, to divine benevolence.
—Noam Shazeer, in GLU Variants Improve Transformer
Keras layers without using matrix multiplications.
This is a Keras based implementation of some layers mentioned in the papers The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits and Scalable MatMul-free Language Modeling.
Getting Started
Get started with using Keras-MML with the introductory pages.
Code Examples
See what Keras-MML can do with these code examples.
Explanations
Read about the inner workings and rationale behind Keras-MML.
API
Documentation for the components that Keras-MML offers.