Topics

Sequence Modeling

Architectures for modeling long ordered data such as text, audio, code, and genomics.

Attention Is All You Need: The Architecture Under Modern AI

The Transformer removed recurrence and convolution from sequence transduction, replacing them with attention and parallel training; almost every modern LLM stands on that move.

Sequence Modeling · Carnegie Mellon University

Mamba: A Serious Attempt to Challenge Attention on Long Sequences

Mamba makes state space models selective, letting them decide what to remember or forget from the input while scaling linearly with sequence length.