Encoder / Decoder

What it is

Encoder-decoder refers to a model architecture with two parts: an encoder that processes input into a representation, and a decoder that generates output from that representation. Some models (like BERT) are encoder-only; others (like GPT) are decoder-only. Translation models typically use both.

Why it matters

Modern LLMs like GPT and Claude are decoder-only, meaning they generate text directly without a separate encoding step. Understanding this architecture helps explain why these models are good at generation but originally required different approaches for tasks like classification or embedding creation.