Models
Large Language ModelLLM
Also known as: language model
An AI system trained on huge amounts of text that generates language by predicting the next piece of text, one step at a time.
A large language model is a computer program that has read an enormous amount of writing, so much that it has learned the patterns of language. When you type something, it answers by guessing the next word, then the next, and so on, until it has written a full reply. Think of it as an incredibly well-read version of the autocomplete on your phone: it does not "look things up" like a search engine, it writes what most likely comes next based on everything it has seen.
A large language model (LLM) is a type of artificial intelligence that produces text. It is trained on a very large collection of written material and learns one deceptively simple skill: given some text, predict the next token (a word or word-fragment). Repeat that prediction over and over and you get sentences, paragraphs and answers.
Why "large"?
The "large" refers to two things: the amount of training material (often a large slice of the public internet, books and code) and the number of internal settings, called parameters, that the model tunes during training, typically billions of them. Past a certain scale, models start to handle tasks they were never explicitly taught, such as translation or summarising, an effect researchers call emergent behaviour.
What they can and cannot do
Because an LLM writes what is statistically likely rather than what is verified, it can sound confident while being wrong, a failure known as a hallucination. Well-known model families include GPT (OpenAI), Claude (Anthropic), Gemini (Google) and Llama (Meta). Most are built on an architecture called the transformer, and each can only "see" a limited amount of text at once, its context window.
An LLM is an autoregressive neural network, almost always transformer-based, that models a probability distribution over a token vocabulary and generates text by repeatedly sampling the next token conditioned on the preceding sequence. Training has two broad phases. First, pretraining: self-supervised next-token prediction over a very large corpus, which is where the bulk of world knowledge and linguistic competence is acquired. The relationship between model size, data and compute is captured by empirical scaling laws (Kaplan et al. 2020; the compute-optimal revision by Hoffmann et al. 2022, "Chinchilla").
Second, alignment: supervised fine-tuning on curated demonstrations followed, often, by reinforcement learning from human feedback (RLHF) to make outputs more helpful and to follow instructions. At inference time, output is shaped by decoding parameters such as temperature and top-p, and is bounded by the context window. The key limitation to keep in mind: the objective is next-token likelihood, not truth, which is why grounding techniques such as retrieval-augmented generation and tool use are used to tie outputs to verifiable sources.