Models
Encoder & Decoder
Also known as: encoder-only, decoder-only
Two building blocks of transformer models; roughly, an encoder "reads and understands" input, a decoder "writes" output. Most chatbots are decoder-only.
Encoder and decoder are two roles inside AI language models. Loosely, an encoder reads and understands input (great for tasks like classifying or searching text), while a decoder generates new text word by word (great for writing and chatting). Today's chatbots are mostly "decoder-only". The distinction is technical, but it explains why different AI models are suited to different jobs.
The original transformer had two parts: - Encoder: processes the whole input at once to build a rich understanding of it. - Decoder: generates output one token at a time, using that understanding.
Three common architectures descend from this: - Encoder-only (e.g. BERT): great for understanding tasks, classification, search, embeddings. - Decoder-only (e.g. GPT, Claude, most LLMs): built for generation; this is what powers chatbots. - Encoder-decoder (e.g. T5): good for input→output tasks like translation and summarising.
The takeaway
Most modern general-purpose chat LLMs are decoder-only, which turned out to scale and generalise very well.
Encoder and decoder denote the two transformer stacks (Vaswani et al. 2017): the encoder applies bidirectional self-attention to build contextual representations of the full input; the decoder uses causal (masked) self-attention plus cross-attention to the encoder to generate autoregressively. Three lineages emerged, encoder-only (BERT; bidirectional, strong for understanding/embeddings, not generative), decoder-only (GPT-style; causal, generative), and encoder-decoder (T5/BART; sequence-to-sequence).
The field largely converged on decoder-only for general-purpose LLMs, which, with sufficient scale and pretraining, match or exceed encoder-decoder performance across tasks (recast as text-to-text) while being simpler and highly scalable. Encoder-only models remain important for retrieval/embedding and classification where a non-generative bidirectional representation is preferable, and encoder-decoders retain niches in conditional generation like translation. The architectural choice encodes an attention/inductive-bias trade-off between bidirectional understanding and autoregressive generation.